PhpDig.net - View Single Post

vonbrocklin · 11-25-2003, 11:09 AM

For some reason, when I run the spider it is kicking back duplicate documents that are not in fact duplicates.

It indexes this:

Code:

mambo104/index.php?option=com_weblinks&Itemid=4

But then kicks this back as a duplicate:

Code:

mambo104/index.php?option=com_weblinks&Itemid=1&catid=2

The first is actually the top level intro page leading into the second page. Both of them should be indexed because they contain different content. Is this due to a problem with the querystring somehow? Just exactly how does phpdig determine what constitutes a duplicate?

11-25-2003, 11:09 AM	#1
vonbrocklin Green Mole Join Date: Sep 2003 Posts: 5	Duplicate Documents Problem... For some reason, when I run the spider it is kicking back duplicate documents that are not in fact duplicates. It indexes this: Code: mambo104/index.php?option=com_weblinks&Itemid=4 But then kicks this back as a duplicate: Code: mambo104/index.php?option=com_weblinks&Itemid=1&catid=2 The first is actually the top level intro page leading into the second page. Both of them should be indexed because they contain different content. Is this due to a problem with the querystring somehow? Just exactly how does phpdig determine what constitutes a duplicate?