not indexing some links

drywall · 09-01-2004, 01:33 PM

I've got Apache configured to some URL rewriting, so links that would be like dir/page.php?variable=value can be rewritten much more simply as dir//value.

Unfortunately, (I suspect due to the double-/) phpdig is not acknowledging these links and following them, leaving portions of the site un-indexed.

Is there something I can tweak in phpdig somewhere to have it index these URLs instead of ignoring them? Or am I going to need to change my Apache rewrite to accomodate phpDig? (which I'm reluctant but ultimately wiling to do if I have to).

Charter · 09-07-2004, 04:27 PM

Hi. Try looking in the phpdigRewriteUrl function.

drywall · 09-08-2004, 02:19 PM

Thanks for telling me where to look. Unfortunately, though I have a tiny bit of familiarity with regular expressions, my competenance isn't such that I can decipher the wizardry going on in phpdigRewriteUrl. What am I looking for and how would I modify it to play nice with my double-slash URLs?

Charter · 09-11-2004, 07:21 AM

The below takes more than one slash and replaces them with one slash. It shows up a couple of times in the phpdigRewriteUrl function so try removing it from the code.

PHP Code:


			
ereg_replace("/+","/",$path)

drywall · 09-13-2004, 08:06 AM

Aha. I feel dumb for not spotting that one, it's pretty blatant. Thank you.

Unfortunately, neither modifying its behavior (I changed "/+" to "//+") nor removing it entirely (in the two places it appears) worked: phpdig still isn't finding those links.

Your help has been dead-on and seems like it should have fixed things, but...

09-01-2004, 01:33 PM	#1
drywall Green Mole Join Date: May 2004 Posts: 25	not indexing some links I've got Apache configured to some URL rewriting, so links that would be like dir/page.php?variable=value can be rewritten much more simply as dir//value. Unfortunately, (I suspect due to the double-/) phpdig is not acknowledging these links and following them, leaving portions of the site un-indexed. Is there something I can tweak in phpdig somewhere to have it index these URLs instead of ignoring them? Or am I going to need to change my Apache rewrite to accomodate phpDig? (which I'm reluctant but ultimately wiling to do if I have to). Last edited by drywall; 09-01-2004 at 01:35 PM.

09-07-2004, 04:27 PM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Try looking in the phpdigRewriteUrl function. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

09-11-2004, 07:21 AM	#4
Charter Head Mole Join Date: May 2003 Posts: 2,539	The below takes more than one slash and replaces them with one slash. It shows up a couple of times in the phpdigRewriteUrl function so try removing it from the code. PHP Code: `ereg_replace("/+","/",$path)` __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Exclude links from indexing, keep text	digger_123	How-to Forum	0	12-20-2006 05:14 AM
Why aren't all links followed when indexing?	Vertikal	Troubleshooting	6	06-26-2005 03:46 AM
not indexing most links	westghent	Troubleshooting	2	11-17-2004 07:04 PM
Indexing links inside hosts	bazarin	How-to Forum	3	02-23-2004 10:35 AM
Indexing javascript links	paulx82	How-to Forum	1	01-18-2004 09:43 AM

09-08-2004, 02:19 PM	#3
drywall Green Mole Join Date: May 2004 Posts: 25	Thanks for telling me where to look. Unfortunately, though I have a tiny bit of familiarity with regular expressions, my competenance isn't such that I can decipher the wizardry going on in phpdigRewriteUrl. What am I looking for and how would I modify it to play nice with my double-slash URLs?

09-13-2004, 08:06 AM	#5
drywall Green Mole Join Date: May 2004 Posts: 25	Aha. I feel dumb for not spotting that one, it's pretty blatant. Thank you. Unfortunately, neither modifying its behavior (I changed "/+" to "//+") nor removing it entirely (in the two places it appears) worked: phpdig still isn't finding those links. Your help has been dead-on and seems like it should have fixed things, but...