Indexing stops after a few pages

Sibona · 04-26-2005, 12:04 AM

Hello,

When I try to update my site, it now stops indexing after a few pages for no reason. Many pages are not indexed yet although they are properly linked.

I noticed in that in the tempspider, some pages have error set to 1. These pages are in an excluded directory and I do not understand why they are even stored in the tempspider. Besides, they are working pages and I do not see why they get error set to 1.

Any idea ?

Cheers,
Nicolas

Charter · 05-03-2005, 11:27 AM

Check out section 6 of the documentation. Also, try changing some of the config file options such as LIMIT_TO_DIRECTORY, PHPDIG_IN_DOMAIN, *_MAX_LIMIT, and *_LIMIT. Without knowing more about the pages you are tring to index, I don't know why you'd be getting ones in the tempspider table. Perhaps you can provide some examples?

04-26-2005, 12:04 AM	#1
Sibona Former Member Join Date: Apr 2005 Posts: 1	Indexing stops after a few pages Hello, When I try to update my site, it now stops indexing after a few pages for no reason. Many pages are not indexed yet although they are properly linked. I noticed in that in the tempspider, some pages have error set to 1. These pages are in an excluded directory and I do not understand why they are even stored in the tempspider. Besides, they are working pages and I do not see why they get error set to 1. Any idea ? Cheers, Nicolas

05-03-2005, 11:27 AM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Check out section 6 of the documentation. Also, try changing some of the config file options such as LIMIT_TO_DIRECTORY, PHPDIG_IN_DOMAIN, _MAX_LIMIT, and _LIMIT. Without knowing more about the pages you are tring to index, I don't know why you'd be getting ones in the tempspider table. Perhaps you can provide some examples? __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Spider stops before all pages are indexed	halide	Troubleshooting	3	07-19-2005 01:26 AM
Spidering Stops, Site is Locked after 7 Pages	nevsie	Troubleshooting	3	03-02-2005 04:52 PM
Indexing stops after 2 seconds	msberryuk	Troubleshooting	3	01-13-2005 06:28 AM
Spidering stops after 6 level 1 pages	Steve	Troubleshooting	3	05-20-2004 05:56 PM
Spider stops indexing (Browser: Done)	rainbow_six	Troubleshooting	2	04-24-2004 09:16 AM