PhpDig.net

PhpDig.net (http://www.phpdig.net/forum/index.php)
-   Troubleshooting (http://www.phpdig.net/forum/forumdisplay.php?f=22)
-   -   Spidering problem please help (http://www.phpdig.net/forum/showthread.php?t=2272)

KaZ 11-30-2005 09:50 AM

Spidering problem please help
 
I am trying to spider certian urls like http://cgi.ebay.com/Sweetie-Pies-Bab...QQcmdZViewItem

and when it tryes to crawl it it adds a / to the end of the url which makes the url a 404 how would i fix this issue.

thanks in advance
KaZ

KaZ 12-05-2005 06:59 AM

This isnt advertising that is not my link it is just an example i am creating a auction search website and need to crawl auction listings from all auction sites only site im having an issue with so far is Ebay the auction listing URl's are weird and if you add a trailing slash to them it makes the link a 404 but it seems to only do it on these urls other urls like http://auctionsitetest.com/auctiondetails.php?id=222 get listed fine with no trailing slash but for some reason only the ebay links that i have encountered so far do this any suggestions please?


All times are GMT -8. The time now is 10:39 PM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.