getting past session protected pages

[theverychap]

hello there,

this question might look similar to a previous question about getting indexes of protected directories, but that didnt help me at all, so...


i have a site, which is about 90% protected material (it is learning material for a college i work for)

basically, i am using php sessions to protect the pages, if a session_is_registered('valid_user') they are able to view the site, i was logged in when i indexed the site using phpdig, although it wont index the 'need to be logged in' pages...

in the documentaion, it says i can add a username and password somewhere so that the spider can access (and therefore index) these pages...

but where do i put these details? i have created an account especially for the spider to access the site, but need to know what to do next...

please help!!
thanks in advance too :)

Stuart

[Charter]

Hi. If your OS allows them, you can set .htaccess and .htpasswd files and then pass the username and password via the URL like so: http://user:pass@www.domain.com/protected/dir/

If you want the search to be accessible only for validated users, you could add the sessions code to the PhpDig files to get around passing a username and password via a URL.

If PhpDig hits a site that is protected by sessions, and PhpDig is not a valid user, then it will index the resulting 'You cannot access this page' text rather than the content itself.

When you do a crawl, you may want to set the following so that the session value is not available in the search results, where PHPSESSID is the name of the session variable:
define('PHPDIG_SESSID_REMOVE',true);
define('PHPDIG_SESSID_VAR','PHPSESSID');

[theverychap]

hi Charter,
thanks for the quick reply, but unfortunately im being a bit hopeless here...

the constants that you told me to set are already set as you say,
i dont really fancy putting the username/pass in the URL either, so th way forward i think is to do as you suggest: put the session code in the phpDig files...

I am including the phpDig index.php (renamed to index_phpdig.php) in my /search/index.php file which itself is including a header and a footer, now, the session code and userCheck that im using is all in the header file, which gives me a very straight-forward user control, that works.

the search is working great for the pages that arent protected, but as you say, it is indexing my 'you arent allowed unless logged in...' bit... so, as far as i am concerned the session code is within the phpDig files...

sorry to sound a bit thick, but would you mind ellaborating on what you said previously (incorporating session code into dig...)

thanks again.

[Charter]

Hi. The files in the admin directory make post or get requests to other files in the admin directory so try adding the session code to all of the PHP files in the admin directory, except for the debug_functions.php and robot_functions.php files. This of course assumes that the session that is set depends on user so that other valid users cannot access PhpDig once logged in.

[theverychap]

hi there,

still no luck im afraid, i just cannot seem to get phpdig to index the pages that i have protected...

thanks for your help though!

i have kind of given up (ooh, the shame..!), a bit more research maybe, or a different way of making a search facility.

cheers,
Stuart.