|
10-23-2006, 11:29 PM | #1 |
Green Mole
Join Date: Apr 2006
Posts: 4
|
Long Time Spidering Crashes
I am spidering a huge web with a big amount of data. Very often, the web spidering (launched from admin) ends unexpectedly ("Page not found") 1, 2 or 3 hours later... Is it normal? Can a phpdig process, for example, work 24h continously?
Thanks |
10-26-2006, 09:25 AM | #2 |
Head Mole
Join Date: May 2003
Posts: 2,539
|
If you are using the admin panel, and spidering via your browser, you might consider using shell instead, especially if you are spidering for hours. Not sure but it might be the case that the browser cannot handle that much text. Plus, spidering from shell should be faster.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension. |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Long Call help. | GunMuse | How-to Forum | 0 | 07-26-2006 10:11 AM |
Long post, possible issues, and maybe the data to solve some? | Paul D. Buck | Troubleshooting | 4 | 02-03-2005 11:59 AM |
How long? | sjSJ | How-to Forum | 1 | 05-11-2004 06:09 PM |
PDFs in directory listed as one long entry | motopsycho | External Binaries | 4 | 03-09-2004 05:07 PM |
Request Time | mongeon | Mod Requests | 1 | 10-05-2003 10:15 AM |