not digging certain links

BulForce · 08-31-2004, 07:00 AM

I try this in file that contanis 1000 links to external domains, BUT the digger index only the page with the 1000 links and the first one of external links. I try all combinations
between the dig depth and links per. But the situation is same

BulForce · 08-31-2004, 07:25 AM

I think that the problem become from the specific of the links
i will give a short example of the links:

http://somewhere.com/path1/path1/file1.php?someid,1,1,1
http://somewhere.com/path2/path2/file2.php?someid,2,2,2
http://somewhere.com/path3/path3/file3.php?someid,3,3,3
http://somewhere.com/path4/path4/file4.php?someid,4,4,4
http://somewhere.com/path5/path5/file5.php?someid,5,5,5

The spider looks at them, then index http://somewhere.com/path2/path2/file2.php?someid, but after that start looking at http://somewhere.com/ and i think that he thinks that all the other links are pointed to same location.

Charter i will be very happy if you can help me

Thanks in advance

BulForce · 09-10-2004, 07:27 AM

Is this thread forgotten by the support ?????

Charter · 09-11-2004, 08:15 AM

What version of PhpDig are you using? What do you get with the following code?

PHP Code:


			
<?php

$url = "http://somewhere.com/path1/path1/file1.php?someid,1,1,1";

print_r(parse_url($url));

?>

BulForce · 09-15-2004, 05:02 PM

Digger version is "PhpDig v.1.8.3"

An the code output is:

Array ( [scheme] => http [host] => somewhere.com [path] => /path1/path1/file1.php [query] => someid,1,1,1 )

I hope that this will help you.

BulForce · 09-19-2004, 07:48 AM

Does anyone have ideas about solution of this problem ?

BulForce · 09-30-2004, 08:15 AM

Charter ? Any idea about resolving this problem.

Charter · 09-30-2004, 11:03 AM

Maybe a redirect issue? I cannot say for sure.

BulForce · 10-03-2004, 06:27 AM

Can you point me to some solution ?

BulForce · 10-11-2004, 10:13 AM

Charter, i found that i have made a mistake when gived this example
Acctually the links are same and only ID is diffrent. Host and path are same example:

http://www.somewhere.com/index.php?id=1,2,0
http://www.somewhere.com/index.php?id=1,3,0
http://www.somewhere.com/index.php?id=1,4,0 and etc.

But this are tottaly diffrent pages, so is there any way to make the spider to index them ?

Thank you

08-31-2004, 07:25 AM	#2
BulForce Orange Mole Join Date: Aug 2004 Location: none Posts: 33	I think that the problem become from the specific of the links i will give a short example of the links: http://somewhere.com/path1/path1/file1.php?someid,1,1,1 http://somewhere.com/path2/path2/file2.php?someid,2,2,2 http://somewhere.com/path3/path3/file3.php?someid,3,3,3 http://somewhere.com/path4/path4/file4.php?someid,4,4,4 http://somewhere.com/path5/path5/file5.php?someid,5,5,5 The spider looks at them, then index http://somewhere.com/path2/path2/file2.php?someid, but after that start looking at http://somewhere.com/ and i think that he thinks that all the other links are pointed to same location. Charter i will be very happy if you can help me Thanks in advance Last edited by BulForce; 08-31-2004 at 08:14 AM.

09-11-2004, 08:15 AM	#4
Charter Head Mole Join Date: May 2003 Posts: 2,539	What version of PhpDig are you using? What do you get with the following code? PHP Code: `<?php $url = "http://somewhere.com/path1/path1/file1.php?someid,1,1,1"; print_r(parse_url($url)); ?>` __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

09-30-2004, 11:03 AM	#8
Charter Head Mole Join Date: May 2003 Posts: 2,539	Maybe a redirect issue? I cannot say for sure. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
No Digging on same server	robertlyman	Troubleshooting	1	07-05-2005 06:22 PM
limit digging to certain keywords?	hgw	How-to Forum	1	08-20-2004 06:57 AM
digging subdomains	b2l_grefix	How-to Forum	6	05-10-2004 03:34 PM
digging probs	michabis101	How-to Forum	3	04-10-2004 12:26 PM

08-31-2004, 07:00 AM	#1
BulForce Orange Mole Join Date: Aug 2004 Location: none Posts: 33	I try this in file that contanis 1000 links to external domains, BUT the digger index only the page with the 1000 links and the first one of external links. I try all combinations between the dig depth and links per. But the situation is same

09-10-2004, 07:27 AM	#3
BulForce Orange Mole Join Date: Aug 2004 Location: none Posts: 33	Is this thread forgotten by the support ?????

09-15-2004, 05:02 PM	#5
BulForce Orange Mole Join Date: Aug 2004 Location: none Posts: 33	Digger version is "PhpDig v.1.8.3" An the code output is: Array ( [scheme] => http [host] => somewhere.com [path] => /path1/path1/file1.php [query] => someid,1,1,1 ) I hope that this will help you.

09-19-2004, 07:48 AM	#6
BulForce Orange Mole Join Date: Aug 2004 Location: none Posts: 33	Does anyone have ideas about solution of this problem ?

09-30-2004, 08:15 AM	#7
BulForce Orange Mole Join Date: Aug 2004 Location: none Posts: 33	Charter ? Any idea about resolving this problem.

10-03-2004, 06:27 AM	#9
BulForce Orange Mole Join Date: Aug 2004 Location: none Posts: 33	Can you point me to some solution ?

10-11-2004, 10:13 AM	#10
BulForce Orange Mole Join Date: Aug 2004 Location: none Posts: 33	Charter, i found that i have made a mistake when gived this example Acctually the links are same and only ID is diffrent. Host and path are same example: http://www.somewhere.com/index.php?id=1,2,0 http://www.somewhere.com/index.php?id=1,3,0 http://www.somewhere.com/index.php?id=1,4,0 and etc. But this are tottaly diffrent pages, so is there any way to make the spider to index them ? Thank you