|
11-17-2006, 09:53 AM | #1 |
Green Mole
Join Date: Nov 2006
Posts: 3
|
Limit of spidering external domains
My first steps with phpdig went well. I spidered a domain and the subs was always indexed. OK,that can i limit in the config. And inthe first test-runs the spider went not outside this (given domain and his subdomains). On the actuall spidering of a website the spider goes outside the domain and indexes the foreign domain with the same settings of depth and links. What i want is
1. Spider the given domain OR Spider the given domain and his subs. 2. Go not outside the given domain(s) I have read the config-file more than once and searched here in the forum but did not found a solution. Thanks for help / answering Vadim |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
spidering external links | websearch | How-to Forum | 1 | 01-11-2005 08:39 AM |
Calling all persons spidering multiple domains | Slider | How-to Forum | 2 | 11-06-2004 01:12 AM |
External link | pirw3t | How-to Forum | 6 | 09-24-2004 03:11 PM |
redirect to external domains | sf44 | Troubleshooting | 4 | 07-03-2004 11:56 PM |
Searching external domains/links | kenazo | How-to Forum | 3 | 03-14-2004 02:55 PM |