Hey man! something mad has happened!
I made a html document, just a big list of these:
<a href="http://www.mysite.com/database/world/uk/england/east_anglia/cambridgeshire/"</a>
for all the counties, regions countries and one for the uk....
and now when I spider it also spiders all the town folders!
The php is being created and spidered, its like its had a kick up the arse and a map as to where to go!!!
HAHAHA fantastic! Any idea why this has happened?
cheers for all the help, much appreciated,
hopefully this will be a trick that other people can use!
Alex.