Charter,
It still doesn't do the job. Main reason is the size of the files I spider and the amount of HTML tags they contain. And since I have a genealogy site, adding newly found ancestors and their descendants will only lead to increasing file sizes. For the time being I have chosen to generate my genealogy files in two flavours. One with complete functionality and the other without any HTML tag. I use the latter for spidering and afterwards replace it with the correct one.
I believe a final solution might be to spider locally and then copy the local database to the database on the remote server. But this will need some investigation
.
For now, thanks for your help.
Peter