Working with Domains

bazarin · 02-26-2004, 02:26 AM

Hi:

I am working with PhpDig and it is working fine. I am doing some tests in order to start to PhpDig some sites.
I was testing a site but would like to PhpDig only some subdirectories. The site is this:
http://dollar.biz.uiowa.edu/~fil/Erico/
I am using depth 20 to go as much as possible inside the site.
The problem is: the http://dollar.biz.uiowa.edu/~fil/Erico/
subdirectory has some pages linked to http://dollar.biz.uiowa.edu and PhpDig start to Dig in this link.
I think that if I am PhpDiging in http://dollar.biz.uiowa.edu/~fil/Erico/
this should be the domain and so anything found linked different from this domain 'http://dollar.biz.uiowa.edu/~fil/Erico/
' should not be included in the database.
Is that possible? Am I doing something wrong?
Any help will pe very appreciated.

Regards,

Wagner Bazarin
www.bazarin.com

Charter · 02-28-2004, 04:28 PM

Hi. You might try setting up a robots.txt file, or after crawling, exclude and/or delete certain directories from the admin panel.

02-26-2004, 02:26 AM	#1
bazarin Green Mole Join Date: Feb 2004 Location: Sao Paulo - Brazil Posts: 8	Working with Domains Hi: I am working with PhpDig and it is working fine. I am doing some tests in order to start to PhpDig some sites. I was testing a site but would like to PhpDig only some subdirectories. The site is this: http://dollar.biz.uiowa.edu/~fil/Erico/ I am using depth 20 to go as much as possible inside the site. The problem is: the http://dollar.biz.uiowa.edu/~fil/Erico/ subdirectory has some pages linked to http://dollar.biz.uiowa.edu and PhpDig start to Dig in this link. I think that if I am PhpDiging in http://dollar.biz.uiowa.edu/~fil/Erico/ this should be the domain and so anything found linked different from this domain 'http://dollar.biz.uiowa.edu/~fil/Erico/ ' should not be included in the database. Is that possible? Am I doing something wrong? Any help will pe very appreciated. Regards, Wagner Bazarin www.bazarin.com

02-28-2004, 04:28 PM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. You might try setting up a robots.txt file, or after crawling, exclude and/or delete certain directories from the admin panel. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Banned Domains	JLutterklas	How-to Forum	0	09-05-2006 11:38 AM
Blocking domains	richwilson	How-to Forum	0	03-29-2006 07:02 AM
Same Hosting but 2 differents domains	MoonC	Troubleshooting	1	07-23-2005 12:28 PM
Newbie on Domains: Yes or No Answer Please :)	new2dev	How-to Forum	1	03-02-2005 12:24 AM
Choosy about domains?	druesome	Troubleshooting	7	04-30-2004 01:39 AM