|
Download -
PHP Reference -
Links -
Demo 1.8.9 RC1 -
API
License - Credits - Copying - Changelog - Documentation |
|
|
02-18-2004, 06:47 PM
|
#1 |
|
Green Mole
Join Date: Feb 2004
Posts: 3
|
sleep(5) in spider.php?
Anybody know why the sleep(5) is in the big loop in spider.php? I took it out and surprisingly saved 5 seconds per file indexed. Besides the obvious consequences of not giving the CPU a rest, is there any good reason to have a sleep?
|
|
|
|
02-19-2004, 06:40 AM
|
#2 |
|
Head Mole
Join Date: May 2003
Posts: 2,539
|
Hi. If you crawl sites that are not yours, setting a delay is the courteous thing to do so that you do not place too high of a load on the other person's machine.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension. |
|
|
|
|
03-12-2004, 02:31 PM
|
#3 |
|
Green Mole
Join Date: Mar 2004
Location: Russia
Posts: 21
|
Why don't make a settings for that? for example "force indexing!"
 And when it's on, indexing is speeded up. Or force every 5 urls and sleep after that for a 5 secs, then again... and so on...
|
|
|
|
 |
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| autorefresh spider.php | zaartix | Mod Requests | 0 | 12-21-2005 10:11 PM |
| Change sleep | noel | How-to Forum | 1 | 10-27-2005 10:58 AM |
| Sleep in the code | davids211082 | How-to Forum | 2 | 05-19-2005 12:22 AM |
| I have one bug with spider.php | Booboo | Troubleshooting | 1 | 03-01-2005 02:46 AM |
| spider.php via bash | tomas | Troubleshooting | 16 | 02-07-2004 05:23 PM |
