|
Timeout setting on 404
I got the problem, that some documents, which are getting 404 on a re-spidering take really minutes (about 7 minutes) before they are recognized as 404. Is there a way to reduce this time, since this is really blocking the whole spidering process. Image 7 minutes for each (not anymore existing) page...
Greetings,
Olaf
|