PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 02-22-2006, 08:38 PM   #1
wineo
Green Mole
 
Join Date: Feb 2006
Posts: 2
Not spidering mod rewritten pages

I have two sub domains for my site and phpdig spiders the one nicely (including mod rewrites). BUT, it is not spidering further than this new page on the other sub domain. It will not go deeper than the page http://www.wineography.com/wine_reviews.htm . Here is the report:

Code:
SITE : http://www.wineography.com/
Exclude paths :
- @NONE@
- cellar_wine_search.htm
- member_details.htm
Duplicate of an existing document
1:http://www.wineography.com/wine_reviews.htm
(time : 00:00:05)
No link in temporary table
links found : 1
http://www.wineography.com/wine_reviews.htm
Optimizing tables...
Indexing complete !
Any ideas on how to force it?!
wineo is offline   Reply With Quote
Old 02-22-2006, 08:53 PM   #2
wineo
Green Mole
 
Join Date: Feb 2006
Posts: 2
I physically indexed one of the mod pages () and it managed to index it and found one other, but still did not continue further. I had the link depth set to 20 and the limit set to 0...

SITE : http://www.wineography.com/
Exclude paths :
- @NONE@
- cellar_wine_search.htm
- member_details.htm
1:http://www.wineography.com/USA/country/212
(time : 00:00:06)
+
level 1...
2:http://www.wineography.com/top_wineries.htm
(time : 00:00:19)

No link in temporary table
links found : 2
http://www.wineography.com/USA/country/212
http://www.wineography.com/top_wineries.htm
Optimizing tables...
Indexing complete !
wineo is offline   Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Mod-rewrite = spidering / URL problems jcnorris Troubleshooting 1 10-26-2006 10:38 AM
Spidering Stops, Site is Locked after 7 Pages nevsie Troubleshooting 3 03-02-2005 04:52 PM
not spidering all pages (too many links on page?) mirdin Troubleshooting 2 09-01-2004 07:08 AM
Spidering stops after 6 level 1 pages Steve Troubleshooting 3 05-20-2004 05:56 PM
Restrict Spidering to new pages only bushpilot How-to Forum 1 05-01-2004 09:07 AM


All times are GMT -8. The time now is 05:19 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.