getting past session protected pages

theverychap · 11-28-2003, 05:25 AM

hello there,

this question might look similar to a previous question about getting indexes of protected directories, but that didnt help me at all, so...

i have a site, which is about 90% protected material (it is learning material for a college i work for)

basically, i am using php sessions to protect the pages, if a session_is_registered('valid_user') they are able to view the site, i was logged in when i indexed the site using phpdig, although it wont index the 'need to be logged in' pages...

in the documentaion, it says i can add a username and password somewhere so that the spider can access (and therefore index) these pages...

but where do i put these details? i have created an account especially for the spider to access the site, but need to know what to do next...

please help!!
thanks in advance too

Stuart

Charter · 11-28-2003, 06:36 AM

Hi. If your OS allows them, you can set .htaccess and .htpasswd files and then pass the username and password via the URL like so: http://user:pass@www.domain.com/protected/dir/

If you want the search to be accessible only for validated users, you could add the sessions code to the PhpDig files to get around passing a username and password via a URL.

If PhpDig hits a site that is protected by sessions, and PhpDig is not a valid user, then it will index the resulting 'You cannot access this page' text rather than the content itself.

When you do a crawl, you may want to set the following so that the session value is not available in the search results, where PHPSESSID is the name of the session variable:
define('PHPDIG_SESSID_REMOVE',true);
define('PHPDIG_SESSID_VAR','PHPSESSID');

theverychap · 11-28-2003, 06:50 AM

hi Charter,
thanks for the quick reply, but unfortunately im being a bit hopeless here...

the constants that you told me to set are already set as you say,
i dont really fancy putting the username/pass in the URL either, so th way forward i think is to do as you suggest: put the session code in the phpDig files...

I am including the phpDig index.php (renamed to index_phpdig.php) in my /search/index.php file which itself is including a header and a footer, now, the session code and userCheck that im using is all in the header file, which gives me a very straight-forward user control, that works.

the search is working great for the pages that arent protected, but as you say, it is indexing my 'you arent allowed unless logged in...' bit... so, as far as i am concerned the session code is within the phpDig files...

sorry to sound a bit thick, but would you mind ellaborating on what you said previously (incorporating session code into dig...)

thanks again.

Charter · 11-28-2003, 11:04 AM

Hi. The files in the admin directory make post or get requests to other files in the admin directory so try adding the session code to all of the PHP files in the admin directory, except for the debug_functions.php and robot_functions.php files. This of course assumes that the session that is set depends on user so that other valid users cannot access PhpDig once logged in.

theverychap · 12-03-2003, 06:18 AM

hi there,

still no luck im afraid, i just cannot seem to get phpdig to index the pages that i have protected...

thanks for your help though!

i have kind of given up (ooh, the shame..!), a bit more research maybe, or a different way of making a search facility.

cheers,
Stuart.

11-28-2003, 05:25 AM	#1
theverychap Green Mole Join Date: Nov 2003 Posts: 3	getting past session protected pages hello there, this question might look similar to a previous question about getting indexes of protected directories, but that didnt help me at all, so... i have a site, which is about 90% protected material (it is learning material for a college i work for) basically, i am using php sessions to protect the pages, if a session_is_registered('valid_user') they are able to view the site, i was logged in when i indexed the site using phpdig, although it wont index the 'need to be logged in' pages... in the documentaion, it says i can add a username and password somewhere so that the spider can access (and therefore index) these pages... but where do i put these details? i have created an account especially for the spider to access the site, but need to know what to do next... please help!! thanks in advance too Stuart

11-28-2003, 06:36 AM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. If your OS allows them, you can set .htaccess and .htpasswd files and then pass the username and password via the URL like so: http://user:pass@www.domain.com/protected/dir/ If you want the search to be accessible only for validated users, you could add the sessions code to the PhpDig files to get around passing a username and password via a URL. If PhpDig hits a site that is protected by sessions, and PhpDig is not a valid user, then it will index the resulting 'You cannot access this page' text rather than the content itself. When you do a crawl, you may want to set the following so that the session value is not available in the search results, where PHPSESSID is the name of the session variable: define('PHPDIG_SESSID_REMOVE',true); define('PHPDIG_SESSID_VAR','PHPSESSID'); __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

11-28-2003, 11:04 AM	#4
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. The files in the admin directory make post or get requests to other files in the admin directory so try adding the session code to all of the PHP files in the admin directory, except for the debug_functions.php and robot_functions.php files. This of course assumes that the session that is set depends on user so that other valid users cannot access PhpDig once logged in. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Indexing Password Protected pages (using session variables)	apetersen	How-to Forum	1	03-27-2007 05:18 AM
Hiding Password Protected Pages	joannelee	How-to Forum	2	03-15-2005 11:07 AM
Indexing cookie/session authenticated pages	tester	Troubleshooting	10	08-18-2004 10:57 AM
Session Newbie	tanbou1	Coding & Tutorials	1	04-13-2004 02:06 PM
Version 1.8.0 and session IDs & links	Charter	Mod Submissions	0	01-21-2004 04:04 PM

11-28-2003, 06:50 AM	#3
theverychap Green Mole Join Date: Nov 2003 Posts: 3	hi Charter, thanks for the quick reply, but unfortunately im being a bit hopeless here... the constants that you told me to set are already set as you say, i dont really fancy putting the username/pass in the URL either, so th way forward i think is to do as you suggest: put the session code in the phpDig files... I am including the phpDig index.php (renamed to index_phpdig.php) in my /search/index.php file which itself is including a header and a footer, now, the session code and userCheck that im using is all in the header file, which gives me a very straight-forward user control, that works. the search is working great for the pages that arent protected, but as you say, it is indexing my 'you arent allowed unless logged in...' bit... so, as far as i am concerned the session code is within the phpDig files... sorry to sound a bit thick, but would you mind ellaborating on what you said previously (incorporating session code into dig...) thanks again.

12-03-2003, 06:18 AM	#5
theverychap Green Mole Join Date: Nov 2003 Posts: 3	hi there, still no luck im afraid, i just cannot seem to get phpdig to index the pages that i have protected... thanks for your help though! i have kind of given up (ooh, the shame..!), a bit more research maybe, or a different way of making a search facility. cheers, Stuart.