indexing problem??

Chris2 · 02-20-2004, 06:53 PM

it's widely discussed around here already... the "just recently indexed problem", now i've followed all the threads, understood only half of it and finally just did two things:

hit "delete site" and "update form" on the admin/index.php page serveral times... (that surely wasn't it)

and:

rewrote my robots.txt:

problem: there was just a single space in fron of "User-Agent:" in the first line...

and see, jump, enjoy:

it's running again...

so to anyone who is getting crazy as well: check the robots.txt syntax, some spider engines may accept minor faults, others don't!

Charter · 02-20-2004, 07:05 PM

Hi. Sounds like it may be a regex issue. What was the original robots.txt file, and what is the new robots.txt index file?

Chris2 · 02-21-2004, 08:23 AM

Hi Charter,

in the old robots.txt I had just a space at the beginning of the first line, now I've changed that and it is working again... so I don't have a prob. anymore but I just reported that so that other people might look into their robtos.txt before they get crazy trying to find out what a "NAT" is... (that's what happened to me when I first encounterted a prob. and read around here)

Keep up your good work, phpdig rocks.

Thanks

Chris2

02-20-2004, 06:53 PM	#1
Chris2 Green Mole Join Date: Jan 2004 Posts: 3	indexing problem?? it's widely discussed around here already... the "just recently indexed problem", now i've followed all the threads, understood only half of it and finally just did two things: hit "delete site" and "update form" on the admin/index.php page serveral times... (that surely wasn't it) and: rewrote my robots.txt: problem: there was just a single space in fron of "User-Agent:" in the first line... and see, jump, enjoy: it's running again... so to anyone who is getting crazy as well: check the robots.txt syntax, some spider engines may accept minor faults, others don't!

02-20-2004, 07:05 PM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Sounds like it may be a regex issue. What was the original robots.txt file, and what is the new robots.txt index file? __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Problem with indexing	Raghavendra	Script Installation	3	09-25-2006 09:52 PM
Indexing problem...	afrim_05	Troubleshooting	3	11-24-2005 04:51 AM
Indexing problem	deshaye7	Troubleshooting	1	06-01-2005 07:57 AM
Yet another indexing problem	sbhikes	Troubleshooting	4	05-17-2004 11:20 AM
indexing problem outside localhost	onlytrue	Troubleshooting	2	03-18-2004 01:46 AM

02-21-2004, 08:23 AM	#3
Chris2 Green Mole Join Date: Jan 2004 Posts: 3	Hi Charter, in the old robots.txt I had just a space at the beginning of the first line, now I've changed that and it is working again... so I don't have a prob. anymore but I just reported that so that other people might look into their robtos.txt before they get crazy trying to find out what a "NAT" is... (that's what happened to me when I first encounterted a prob. and read around here) Keep up your good work, phpdig rocks. Thanks Chris2