View Single Post
Old 11-17-2006, 09:53 AM   #1
Vadim
Green Mole
 
Join Date: Nov 2006
Posts: 3
Limit of spidering external domains

My first steps with phpdig went well. I spidered a domain and the subs was always indexed. OK,that can i limit in the config. And inthe first test-runs the spider went not outside this (given domain and his subdomains). On the actuall spidering of a website the spider goes outside the domain and indexes the foreign domain with the same settings of depth and links. What i want is
1. Spider the given domain
OR Spider the given domain and his subs.
2. Go not outside the given domain(s)

I have read the config-file more than once and searched here in the forum but did not found a solution.

Thanks for help / answering

Vadim
Vadim is offline   Reply With Quote