Reindexing site won't index certain page

gman · 08-06-2004, 10:03 AM

I successfully indexed my site. Later, I came back to reindex the new files I added. That did not work. It acts like it doesn't see the new files.

It keeps saying "Duplicate of existing document" with a red x on the left.

I think that it skips the desired page because it is considering all my pages duplicates, so it won't link to the new page.

How do I index a single page? I have read through the forums with no success.

If I type in the URI of the new page, it will not index it from the web interface. I have tried many different combinations of search depths and link depths.

Any ideas??

Charter · 08-06-2004, 10:35 AM

Hi. Are there links to the new files? What is LIMIT_DAYS set to in the config file? Perhaps this thread might help.

gman · 08-06-2004, 11:50 AM

When I couldn't index my newly created page, I tried deleting the branch in hopes that I could just reindex the branch and the new page would be picked up, but that didn't work.

LIMIT_DAYS is set to 0.

I have a page that links to the newly created page.
I have tried indexing them both with no success.

I'm using the version 1.8.3.

I tried what was posted in the thread. The links were displayed correctly on the test page I made.

I tried to index that test page's URI, but with no success.

I tried search depth 1, link depth 7. Because there were seven links on the test page.

I also tried search depth 7, link depth 0.

Still don't know what is wrong.

Charter · 08-06-2004, 12:56 PM

Try setting error_reporting(E_ALL); at the top of the spider.php file and see if it throws any errors.

gman · 08-06-2004, 01:05 PM

I think I found the problem.
I had a robots.txt file in the root of my site that disallows spiders.

It's indexing as we speak.

Thank you for the help.

It works now.

You are the "Head Mole"!

08-06-2004, 10:03 AM	#1
gman Green Mole Join Date: Dec 2003 Location: Texas Posts: 7	Reindexing site won't index certain page I successfully indexed my site. Later, I came back to reindex the new files I added. That did not work. It acts like it doesn't see the new files. It keeps saying "Duplicate of existing document" with a red x on the left. I think that it skips the desired page because it is considering all my pages duplicates, so it won't link to the new page. How do I index a single page? I have read through the forums with no success. If I type in the URI of the new page, it will not index it from the web interface. I have tried many different combinations of search depths and link depths. Any ideas??

08-06-2004, 10:35 AM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Are there links to the new files? What is LIMIT_DAYS set to in the config file? Perhaps this thread might help. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

08-06-2004, 12:56 PM	#4
Charter Head Mole Join Date: May 2003 Posts: 2,539	Try setting error_reporting(E_ALL); at the top of the spider.php file and see if it throws any errors. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
cannot index my site	ENTHALPIE	Troubleshooting	2	11-18-2005 02:02 AM
Cron reindexing not working	moreorless	Troubleshooting	2	10-02-2005 10:23 PM
Reindexing non-existing urls	AllKnightAccess	Troubleshooting	0	11-06-2004 11:31 AM
How to Index a site?	davey147	Troubleshooting	0	08-30-2004 12:10 PM
Automatic reIndexing - How to set up	2440media	How-to Forum	2	06-17-2004 05:01 AM

08-06-2004, 11:50 AM	#3
gman Green Mole Join Date: Dec 2003 Location: Texas Posts: 7	When I couldn't index my newly created page, I tried deleting the branch in hopes that I could just reindex the branch and the new page would be picked up, but that didn't work. LIMIT_DAYS is set to 0. I have a page that links to the newly created page. I have tried indexing them both with no success. I'm using the version 1.8.3. I tried what was posted in the thread. The links were displayed correctly on the test page I made. I tried to index that test page's URI, but with no success. I tried search depth 1, link depth 7. Because there were seven links on the test page. I also tried search depth 7, link depth 0. Still don't know what is wrong.

08-06-2004, 01:05 PM	#5
gman Green Mole Join Date: Dec 2003 Location: Texas Posts: 7	I think I found the problem. I had a robots.txt file in the root of my site that disallows spiders. It's indexing as we speak. Thank you for the help. It works now. You are the "Head Mole"!