How to index a directory with pdf files

simonced · 02-11-2004, 06:54 AM

Hello everybody,

I didn't find answer to this question, I post it here in details :
I want to index a folder on my site wich contents only pdf files.
These files are added by an admin page, and I want these pdf files to be findable with phpdig.
(launching once a day with a cron an indexing new or changed files)
so, I don't want to index by http way, but, by the filesystem way.
(Not ftp...)
Is it possible ?

Thanks a lot by advance.

Charter · 02-11-2004, 07:05 AM

Hi. You could make one filename.html that links to the PDF files that you want crawled, and index filename.html at level one. After the index is done, just go to the admin panel, click a site, click the update button, click a blue arrow, and delete the filename.html on the right hand side if you don't want it to show in the search results.

simonced · 02-11-2004, 11:28 PM

Thanks for your so quick reply

I see, it's a good way.
So, I think my file that lists the pdf can be a php script ?
So, it's easy in fact.

How can I crawl only my pdf listing file in shell command ?
$ php [...]/spider.php http://website/fold1/...../foldx/listingfiles.php
(I don't know how to set the level at one this way...)
May I put a robots.txt in the foldx ?

Where can I get a robots configuration help ?

Thank you very much.

Charter · 02-13-2004, 10:41 AM

Hi. To get a search depth of one for indexing from shell, set the following in the config.php file:

PHP Code:


			
define('SPIDER_MAX_LIMIT',1);           //max recurse levels in spider

define('SPIDER_DEFAULT_LIMIT',1);       //default value

define('RESPIDER_LIMIT',1);             //recurse limit for update

02-11-2004, 06:54 AM	#1
simonced Green Mole Join Date: Feb 2004 Location: france Posts: 6	How to index a directory with pdf files Hello everybody, I didn't find answer to this question, I post it here in details : I want to index a folder on my site wich contents only pdf files. These files are added by an admin page, and I want these pdf files to be findable with phpdig. (launching once a day with a cron an indexing new or changed files) so, I don't want to index by http way, but, by the filesystem way. (Not ftp...) Is it possible ? Thanks a lot by advance.

02-11-2004, 07:05 AM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. You could make one filename.html that links to the PDF files that you want crawled, and index filename.html at level one. After the index is done, just go to the admin panel, click a site, click the update button, click a blue arrow, and delete the filename.html on the right hand side if you don't want it to show in the search results. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

02-13-2004, 10:41 AM	#4
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. To get a search depth of one for indexing from shell, set the following in the config.php file: PHP Code: `define('SPIDER_MAX_LIMIT',1); //max recurse levels in spider define('SPIDER_DEFAULT_LIMIT',1); //default value define('RESPIDER_LIMIT',1); //recurse limit for update` __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
How to index only local directory files?	sf44	How-to Forum	0	01-28-2005 02:56 AM
Indexing a directory with many files	rafarspd	How-to Forum	2	12-21-2004 01:49 AM
Can only index files in a single directory	gcrachel	Troubleshooting	5	09-28-2004 06:23 AM
Not indexing pdf files	jayhawk	External Binaries	13	02-18-2004 05:13 AM
Search PDF files	chazter	External Binaries	4	10-02-2003 06:47 AM

02-11-2004, 11:28 PM	#3
simonced Green Mole Join Date: Feb 2004 Location: france Posts: 6	Thanks for your so quick reply I see, it's a good way. So, I think my file that lists the pdf can be a php script ? So, it's easy in fact. How can I crawl only my pdf listing file in shell command ? $ php [...]/spider.php http://website/fold1/...../foldx/listingfiles.php (I don't know how to set the level at one this way...) May I put a robots.txt in the foldx ? Where can I get a robots configuration help ? Thank you very much.