PhpDig.net

PhpDig.net (http://www.phpdig.net/forum/index.php)
-   How-to Forum (http://www.phpdig.net/forum/forumdisplay.php?f=33)
-   -   Indexing a directory with many files (http://www.phpdig.net/forum/showthread.php?t=1658)

rafarspd 12-19-2004 06:46 AM

Indexing a directory with many files
 
This old boy's brain :confused: has stopped functioning - HELP please.

Original version was 1.8.0.
PHPDIG was removed from the PHP area and the SQL data dropped.
I started afresh with V1.8.6.
My problem was only apparent after installation of the latest version.

I have a sub-directory which contains 79 *.txt files (2K to 13K each) which are sequentially numbered. [Newsletters, generated and uploaded monthly].
The search results only display 17 of them and it appears that they are not in number order but in the order stored on the ISP's HDD.

I have changed the following (as an experiment):

define('SPIDER_MAX_LIMIT',50);
define('RESPIDER_LIMIT',50);
define('LINKS_MAX_LIMIT',50);
define('RELINKS_LIMIT',50);

with no difference.

Obviously I am looking in the wrong area so any ideas!

Charter 12-19-2004 09:02 AM

Set "search depth" to a large number, set "links per" to zero, set LIMIT_TO_DIRECTORY to false.

rafarspd 12-21-2004 01:49 AM

Thanks, it help a bit but I now have to confess to the main reason for the problem.

I forgot a basic principle ? :bang:
If you want PHPDIG to find a file then it must have a URL somewhere pointing to it.

I had removed some references to older text files from a page.

Problem is now cured by adding the URL's (with no text) at the bottom of the html just above </body></html>.
They are not seen when viewing but PHPDIG finds and indexes them.


All times are GMT -8. The time now is 08:02 PM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.