Prevent search in some folders

wessam · 08-06-2004, 04:13 PM

dear all
we can Prevent search in some folders with phpdig search (for example" never search and indexing (images , _???, admin \ ) folders ?????????

vinyl-junkie · 08-06-2004, 06:53 PM

If you're on a Linux server, just set your robots.txt to exclude those folders. Another option is to click the red "x" to exclude those directories from the index after spidering, which also deletes them from the index.

Hope this helps.

wessam · 08-06-2004, 11:15 PM

what about solaris server same thing only set robots.txt exclude

Charter · 08-07-2004, 12:07 PM

Hi. Just stick a robots.txt file in web root. PhpDig will look for the file at http://www.domain.com/robots.txt but if you already crawled directories that you want to exclude, then you can delete and/or exclude them from the PhpDig admin panel. With version 1.8.2+ you can also limit the index to a specific directory. If you have PhpDig crawl http://www.domain.com/dir/ (with ending slash) then it should only crawl content that resides in that directory.

wessam · 08-07-2004, 03:38 PM

Sorry but i can't inderstand i must creat robots.txt file or what .
if i must creat it how and i would like see an example
thanks ..
any way i did some modification on the search engine now i have advance search section (search by (Meta Tags "Keyword,title,author,description") and by date (week-6 month or from to date)
i can post this new modification or not ?
thanks

vinyl-junkie · 08-07-2004, 03:55 PM

Here is a site I usually point out to people who want to know something about using a robots.txt file on their site. Hope it helps.

As to the modification you have, I'm sure Charter wouldn't mind if you posted it. There is no guarantee it would end up in the next release, but perhaps someone may find it useful.

wessam · 08-07-2004, 11:16 PM

Thank you

08-06-2004, 04:13 PM	#1
wessam Orange Mole Join Date: Jul 2004 Posts: 30	Prevent search in some folders dear all we can Prevent search in some folders with phpdig search (for example" never search and indexing (images , _???, admin \ ) folders ?????????

08-07-2004, 12:07 PM	#4
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Just stick a robots.txt file in web root. PhpDig will look for the file at http://www.domain.com/robots.txt but if you already crawled directories that you want to exclude, then you can delete and/or exclude them from the PhpDig admin panel. With version 1.8.2+ you can also limit the index to a specific directory. If you have PhpDig crawl http://www.domain.com/dir/ (with ending slash) then it should only crawl content that resides in that directory. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Search returns comments...how to prevent	csabot	Troubleshooting	0	06-07-2006 12:47 PM

08-06-2004, 06:53 PM	#2
vinyl-junkie Purple Mole Join Date: Jan 2004 Posts: 694	If you're on a Linux server, just set your robots.txt to exclude those folders. Another option is to click the red "x" to exclude those directories from the index after spidering, which also deletes them from the index. Hope this helps.

08-06-2004, 11:15 PM	#3
wessam Orange Mole Join Date: Jul 2004 Posts: 30	what about solaris server same thing only set robots.txt exclude

08-07-2004, 03:38 PM	#5
wessam Orange Mole Join Date: Jul 2004 Posts: 30	Sorry but i can't inderstand i must creat robots.txt file or what . if i must creat it how and i would like see an example thanks .. any way i did some modification on the search engine now i have advance search section (search by (Meta Tags "Keyword,title,author,description") and by date (week-6 month or from to date) i can post this new modification or not ? thanks

08-07-2004, 03:55 PM	#6
vinyl-junkie Purple Mole Join Date: Jan 2004 Posts: 694	Here is a site I usually point out to people who want to know something about using a robots.txt file on their site. Hope it helps. As to the modification you have, I'm sure Charter wouldn't mind if you posted it. There is no guarantee it would end up in the next release, but perhaps someone may find it useful.

08-07-2004, 11:16 PM	#7
wessam Orange Mole Join Date: Jul 2004 Posts: 30	Thank you