version 1.8.2, 1.8.3 doesn't fully spidering the site

[Siava]

Version 1.8.0 worked very good! Any days before I see the new version 1.8.2, download and install for the test.
But this version doesn't fully spidering my site during first spidering process.
1.8.3 has this problem to =(

in config.php I change:

define('ABSOLUTE_SCRIPT_PATH','e:\www\search21')
(because my server in the LAN on Windows 2000)

define('PHPDIG_ENCODING','windows-1251'); (because I'm russian =))

define('CHUNK_SIZE',2048);
define('SUMMARY_LENGTH',50000);
define('USE_IS_EXECUTABLE_COMMAND','0');
(this three strings succesfully works with 1.8.0)

@ini_set("magic_quotes_runtime",false);
@ini_set("magic_quotes_sybase",false);
(read in this forum, but changes doesn't make effect)

Server Apache 1.31, php 4.3.8, mysql 4.0.20a

[Siava]

Spidering only root folder and index files of subfolders =(

[pbpub]

I have the same problem. What a pity!

[Siava]

Changes to define('LIMIT_TO_DIRECTORY',false); doesn't make effect =(

[Charter]

What do you use for seach depth and links per?

[pbpub]

seach depth = 4
links per= 0

[lolodev]

why does you not use links per = 4 ?

[zevince]

it seems i have the same problem..
i was using phpdig 1.6.4, and my updates doesnt indexes all the site.. So i've upgraded to version 1.8.3, and tried again..
i've had problems with authentification, wich i finally replaced with a htaccess file (it's a detail). i re-run the spider.php, but it stop without finishing all the site.. i'm retrying to spider right now.... (2nd try with v1.8.3), and i'll tell you the result.. (maybee on monday, rather than now.. my weekend is near!)

[zevince]

oups, i've let the search depth and links per to 0

[zevince]

hmm.. finally i'm still at work, and the spidering stopped..
454 links, and brutally stopped..
hmmmmm... what can i do ? empty the databases and restart spidering ? or what ?


I'm trying right now to index by command line, on the server, directly. i'll tell you monday the result.

[Siava]

Originally posted by Charter
What do you use for seach depth and links per?

15 and 15

[Charter]

Hi. Download the attachment in this post and replace your files with the ones in the ZIP file. Please remember that 'links per' set to anything other than zero can limit the number of links found. Also 'search depth' trumps 'links per' when indexing, so the max number of links found is ((search depth * links per) + 1) when 'links per' is not zero.

[Siava]

Charter
For full spidering I must set 'search depth' and 'links per' to zero? I'm understand right?

Thank you for help I'm going test =)

No, with zero parameters is spiderong only one page =(

[Charter]

Hi. For full spider set 'search depth' to a large value and set 'links per' to zero.

[Siava]

Originally posted by Charter
Hi. For full spider set 'search depth' to a large value and set 'links per' to zero.

Worked! =)))

Thank you for help!