Doesn't index at all!

Ally · 02-04-2005, 08:18 AM

I installed PhpDig on our company's intranet site. Installation was successful but when i try to index site (with depth 10 and links 10, for example) i'm getting

Code:

Spidering in progress... [Stop spider]
Optimizing tables...
Indexing complete !

and that's all! It doesn't index and doesn't give any diagnosis!! Site's url looks like subdomain.ourdomain.tld:7780.
What can be the reason?
Thanks in advance.

clueless · 02-04-2005, 12:40 PM

Hi Ally,

I have had that problem as well I tried it again by adjusting the search depth and the links per that fixed it for me. Give it a try I.e set depth to 1 and links per to 10

jmitchell · 02-04-2005, 01:08 PM

how many times did you try this?

I'm not sure about why it would do this on the first time, but on subsiquent tries this might come up because you recently tried spidering it.

JMitchell

Ally · 02-05-2005, 05:43 PM

I tried it several times with different search depth and the links per but it didn't even appeared in my access/error logs! Does it mean that it couldn't connect to the server?
Thanks.

jmitchell · 02-05-2005, 08:38 PM

try deleting the site from the site list in phpdig, and then try to reindex it.

Charter · 02-07-2005, 01:31 AM

One of the first requests PhpDig makes is for a robots.txt file. If you don't even see this in your server access logs, then maybe the intranet is blocking PhpDig, or maybe you didn't set the correct permissions?

Ally · 02-07-2005, 07:57 AM

The site doesn't have robots.txt at all. Even if a add one with allow / - nothing changes.

The site is accessible from the Internet. How can it be blocked from spiders? I'm desperate! I even doesn't know what do i have to ask my admin about...
Help me, please!

Charter · 02-07-2005, 10:37 AM

Do you see any requests in the server access log from PhpDig? If not, then ask your Intranet admin, if s/he is blocking PhpDig from accessing the Internet, and thus your site.

Ally · 02-07-2005, 03:36 PM

Thanks for your advice, i'll ask him about it.

02-04-2005, 08:18 AM	#1
Ally Green Mole Join Date: Feb 2005 Posts: 4	Doesn't index at all! I installed PhpDig on our company's intranet site. Installation was successful but when i try to index site (with depth 10 and links 10, for example) i'm getting Code: Spidering in progress... [Stop spider] Optimizing tables... Indexing complete ! and that's all! It doesn't index and doesn't give any diagnosis!! Site's url looks like subdomain.ourdomain.tld:7780. What can be the reason? Thanks in advance.

02-04-2005, 12:40 PM	#2
clueless Green Mole Join Date: Feb 2005 Location: usa Posts: 9	Hi Ally, I have had that problem as well I tried it again by adjusting the search depth and the links per that fixed it for me. Give it a try I.e set depth to 1 and links per to 10 __________________ I wish I knew!

02-04-2005, 01:08 PM	#3
jmitchell Orange Mole Join Date: Dec 2004 Location: Tennessee Posts: 60	how many times did you try this? I'm not sure about why it would do this on the first time, but on subsiquent tries this might come up because you recently tried spidering it. JMitchell __________________ 60,000 pages indexed!!!!! http://www.sharemylink.com

02-05-2005, 08:38 PM	#5
jmitchell Orange Mole Join Date: Dec 2004 Location: Tennessee Posts: 60	try deleting the site from the site list in phpdig, and then try to reindex it. __________________ 60,000 pages indexed!!!!! http://www.sharemylink.com

02-07-2005, 01:31 AM	#6
Charter Head Mole Join Date: May 2003 Posts: 2,539	One of the first requests PhpDig makes is for a robots.txt file. If you don't even see this in your server access logs, then maybe the intranet is blocking PhpDig, or maybe you didn't set the correct permissions? __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

02-05-2005, 05:43 PM	#4
Ally Green Mole Join Date: Feb 2005 Posts: 4	I tried it several times with different search depth and the links per but it didn't even appeared in my access/error logs! Does it mean that it couldn't connect to the server? Thanks.

02-07-2005, 07:57 AM	#7
Ally Green Mole Join Date: Feb 2005 Posts: 4	The site doesn't have robots.txt at all. Even if a add one with allow / - nothing changes. The site is accessible from the Internet. How can it be blocked from spiders? I'm desperate! I even doesn't know what do i have to ask my admin about... Help me, please!

02-07-2005, 10:37 AM	#8
Charter Head Mole Join Date: May 2003 Posts: 2,539	Do you see any requests in the server access log from PhpDig? If not, then ask your Intranet admin, if s/he is blocking PhpDig from accessing the Internet, and thus your site. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

02-07-2005, 03:36 PM	#9
Ally Green Mole Join Date: Feb 2005 Posts: 4	Thanks for your advice, i'll ask him about it.