|
Judging by my results a lot would depend on how deep you spider the other domains because the MSQL file on your host would grow quite large and as an estimate, I would imagine that around 500 domains would need around 40mb of MYSQL space to store the data, plus a few megabytes of space for the files and that is based on spidering to a depth of around three, picking up say ten linked files per domain.
Spidering speed varies a lot depending on the quality of the host that your spidering and it can change because of the amount of data flying around the web.
In one of the other forums a guy is offering a prebuilt database so you could contact him, to find out the results he has got from using PHPDIG to spider multiple domains he may well have a good idea what kind of sizes and storage space you may need and PHPDIG's speed over heaps of domains.
PHPDIG as software really is brilliant and does what it is supposed to do and the help and support offered in these forums is brilliant.
I hope that this helps you..
Many regards
Dave A
|