cannot spidering

sderossi · 11-08-2007, 12:08 PM

Hello, I am using Dig R1.8.8 for the first time and I must say the setup/installation was quick and painless.

However now that I am trying to use the application I am experiencing some pain.

After entering the desired URI into the field box and trying to 'Dig it' I either get the messages...

Spidering in progress... [Stop spider]
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------
[Back] to admin interface.

good to see that it worked but it found nothing.
Just the Host table in the admin panel got increment

any help would be greatly appreciated.

sderossi · 11-08-2007, 02:25 PM

Hello,

I have previously gotten phpdig working on our development Windows server that I use here. It's installed, it indexes fine, all is well on the development box.

The problem now is that I am trying to set up phpdig on the production server which is hosted by another company. It's a windows 2003 server (not load balanced, so they tell me), running MySQL 4.1.7, PHP 5.0.4, I'm trying to use phpdig 1.8.8. The phpdig search engine installs fine, no problems. But when I try to index (any URI, anywhere) I just get:

Code:
Spidering in progress... [Stop spider]
SITE : http://www2.anmc.org.au/
Exclude paths :
- @NONE@
1:http://www2.anmc.org.au/
(time : 00:00:00)
No link in temporary table
links found : 1
http://www2.anmc.org.au/
Optimizing tables...
Indexing complete ! [Back] to admin interface.

I go back to the admin index and it shows:

Code:
DataBase status
Hosts : 1 Entries
Pages : 0 Entries
Index : 0 Entries
Keywords : 0 Entries
Temporary table : 0 Entries

This is the same for any website I try to spider, not just the one phpdig is installed on. So I guess this means it's not an issue with a load balanced server, as it does the same thing with any URI I give it.

I have checked that safe_mode is off and allow_url_fopen is on...

I have read other posts on this forum and made sure that PHPDIG_IN_DOMAIN is true, and LIMIT_TO_DIRECTORY is false. I have then tried to spider with depth set to 20 and links per set to 0... but I still get the same results.

Please help.

11-08-2007, 12:08 PM	#1
sderossi Green Mole Join Date: Oct 2007 Posts: 3	cannot spidering Hello, I am using Dig R1.8.8 for the first time and I must say the setup/installation was quick and painless. However now that I am trying to use the application I am experiencing some pain. After entering the desired URI into the field box and trying to 'Dig it' I either get the messages... Spidering in progress... [Stop spider] Optimizing tables... Indexing complete ! -------------------------------------------------------------------------------- [Back] to admin interface. good to see that it worked but it found nothing. Just the Host table in the admin panel got increment any help would be greatly appreciated.

11-08-2007, 02:25 PM	#2
sderossi Green Mole Join Date: Oct 2007 Posts: 3	Not indexing any site Hello, I have previously gotten phpdig working on our development Windows server that I use here. It's installed, it indexes fine, all is well on the development box. The problem now is that I am trying to set up phpdig on the production server which is hosted by another company. It's a windows 2003 server (not load balanced, so they tell me), running MySQL 4.1.7, PHP 5.0.4, I'm trying to use phpdig 1.8.8. The phpdig search engine installs fine, no problems. But when I try to index (any URI, anywhere) I just get: Code: Spidering in progress... [Stop spider] SITE : http://www2.anmc.org.au/ Exclude paths : - @NONE@ 1:http://www2.anmc.org.au/ (time : 00:00:00) No link in temporary table links found : 1 http://www2.anmc.org.au/ Optimizing tables... Indexing complete ! [Back] to admin interface. I go back to the admin index and it shows: Code: DataBase status Hosts : 1 Entries Pages : 0 Entries Index : 0 Entries Keywords : 0 Entries Temporary table : 0 Entries This is the same for any website I try to spider, not just the one phpdig is installed on. So I guess this means it's not an issue with a load balanced server, as it does the same thing with any URI I give it. I have checked that safe_mode is off and allow_url_fopen is on... I have read other posts on this forum and made sure that PHPDIG_IN_DOMAIN is true, and LIMIT_TO_DIRECTORY is false. I have then tried to spider with depth set to 20 and links per set to 0... but I still get the same results. Please help.