|
When I spidered my site recently around noon that day, it took over 2 hours to spider about 1,500 pages. I re-spidered the same site close to midnight that same day, and it took about 45 minutes. I tend to think that during lighter traffic times on your website, the spidering process would be faster. Just a guess though.
|