Indexing stops after a few pages
 

Hello,

When I try to update my site, it now stops indexing after a few pages for no reason. Many pages are not indexed yet although they are properly linked.

I noticed in that in the tempspider, some pages have error set to 1. These pages are in an excluded directory and I do not understand why they are even stored in the tempspider. Besides, they are working pages and I do not see why they get error set to 1.

Any idea ?

Cheers,
Nicolas

Check out section 6 of the documentation. Also, try changing some of the config file options such as LIMIT_TO_DIRECTORY, PHPDIG_IN_DOMAIN, *_MAX_LIMIT, and *_LIMIT. Without knowing more about the pages you are tring to index, I don't know why you'd be getting ones in the tempspider table. Perhaps you can provide some examples?