First crawl works, then subsequent crawls go on forever

andyt · 03-27-2006, 12:56 AM

Hi,

I've got phpDig working on one site and it's fantastic. But I'm having problems setting it up on a different site.

The first time I crawl the site it works no problem but the next time I try, either through the admin interface or the command line, the spider just goes on and on examining the pages. I've had it running for half an hour before I stopped it once. The first crawl only takes a few minutes.

The site I'm indexing has the same menu on every page and I'm setting the search depth to 2 and link per to zero (check all).

Can anyone suggest what the problem might be?

Thanks

Andy

03-27-2006, 12:56 AM	#1
andyt Green Mole Join Date: Mar 2006 Posts: 1	First crawl works, then subsequent crawls go on forever Hi, I've got phpDig working on one site and it's fantastic. But I'm having problems setting it up on a different site. The first time I crawl the site it works no problem but the next time I try, either through the admin interface or the command line, the spider just goes on and on examining the pages. I've had it running for half an hour before I stopped it once. The first crawl only takes a few minutes. The site I'm indexing has the same menu on every page and I'm setting the search depth to 2 and link per to zero (check all). Can anyone suggest what the problem might be? Thanks Andy

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
command line indexing that actually works	carlaron	Troubleshooting	0	11-06-2006 09:48 PM
pdftotext works! but dies after a few pages	bmickler	External Binaries	0	04-05-2006 07:18 PM
Student who try to works with Msword!	davids211082	External Binaries	1	03-15-2005 10:09 AM
How phpdig works with composed keywords ?	julien	How-to Forum	3	03-02-2005 12:12 AM
Indexing doesn't works properly	Alysum	Troubleshooting	7	12-16-2004 08:53 PM