PhpDig.net - View Single Post

pager · 01-15-2004, 11:07 AM

Hello, I'm trying to set up phpdig for a web site and I can make it spider other web sites except mine.

I have tried both locally from the command line and remotely from another server.

Any time I try to spider it the web page freezes for about 30 seconds after I click on the "Dig This!" button and then goes to the result page with:

Spidering in progress...

SITE : http://dev.videx.com/
Exclude paths :
- @NONE@
No link in temporary table

links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !
[Back] to admin interface.

The site is, if you didn't notice

, dev.videx.com and I have managed to spider other servers in our domain (like www.videx.com).

I have removed the robots.txt file from the site but still have a .htaccess restricting use of the /search folder, but otherwise the site is a basic CSS / php based one on a Mac OS X 10.3 server and I am using phpdig version 1.6.2.

I have modified my config file to not search through .css files, but still no luck.

Any suggestions?

01-15-2004, 11:07 AM	#1
pager Green Mole Join Date: Jan 2004 Location: Oregon, USA Posts: 6	Spidering issue with my site Hello, I'm trying to set up phpdig for a web site and I can make it spider other web sites except mine. I have tried both locally from the command line and remotely from another server. Any time I try to spider it the web page freezes for about 30 seconds after I click on the "Dig This!" button and then goes to the result page with: Spidering in progress... SITE : http://dev.videx.com/ Exclude paths : - @NONE@ No link in temporary table links found : 0 ...Was recently indexed Optimizing tables... Indexing complete ! [Back] to admin interface. The site is, if you didn't notice , dev.videx.com and I have managed to spider other servers in our domain (like www.videx.com). I have removed the robots.txt file from the site but still have a .htaccess restricting use of the /search folder, but otherwise the site is a basic CSS / php based one on a Mac OS X 10.3 server and I am using phpdig version 1.6.2. I have modified my config file to not search through .css files, but still no luck. Any suggestions?