specific links not spider

flanders · 10-07-2004, 12:02 AM

hi,

i have a problem! how i can say phpdig, that he dont spider specific links. with :
 and 

it dosnt work. he spider the "text" not, but the link

have anyone an idea?

Charter · 10-07-2004, 12:12 AM

The phpdigExclude and phpdigInclude tags work like this:

Code:

<html>
<body>
<a href="http://www.this_link.com/is_followed.html">This text is indexed.</a>
This text is indexed.
<!-- phpdigExclude -->
<a href="http://www.this_link.com/is_followed.html">This text is ignored.</a>
This text is ignored.
<!-- phpdigInclude -->
<a href="http://www.this_link.com/is_followed.html">This text is indexed.</a>
This text is indexed.
</body>
</html>

Note that the phpdigExclude and phpdigInclude tags need to be on their own lines. If dynamically including a portion of a page, you may need to add a \n newline before and/or after the phpdigExclude and phpdigInclude tags, so that in the HTML source, the tags appear on their own lines.

To exclude specific links from being indexed, try using a robots.txt file, editing FORBIDDEN_EXTENSIONS in the config file, using a no index no follow META tag, or crawl the links and then exclude directories from the admin panel. The FORBIDDEN_EXTENSIONS option requires some regex knowledge.

10-07-2004, 12:02 AM	#1
flanders Green Mole Join Date: Oct 2004 Location: germany Posts: 4	specific links not spider hi, i have a problem! how i can say phpdig, that he dont spider specific links. with : <!-- phpdigExclude --> and <!-- phpdigInclude --> it dosnt work. he spider the "text" not, but the link have anyone an idea? __________________ www.dymny.de

10-07-2004, 12:12 AM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	The phpdigExclude and phpdigInclude tags work like this: Code: <html> <body> <a href="http://www.this_link.com/is_followed.html">This text is indexed.</a> This text is indexed. <!-- phpdigExclude --> <a href="http://www.this_link.com/is_followed.html">This text is ignored.</a> This text is ignored. <!-- phpdigInclude --> <a href="http://www.this_link.com/is_followed.html">This text is indexed.</a> This text is indexed. </body> </html> Note that the phpdigExclude and phpdigInclude tags need to be on their own lines. If dynamically including a portion of a page, you may need to add a \n newline before and/or after the phpdigExclude and phpdigInclude tags, so that in the HTML source, the tags appear on their own lines. To exclude specific links from being indexed, try using a robots.txt file, editing FORBIDDEN_EXTENSIONS in the config file, using a no index no follow META tag, or crawl the links and then exclude directories from the admin panel. The FORBIDDEN_EXTENSIONS option requires some regex knowledge. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
How do I spider list of specific domains?	rich	How-to Forum	0	09-06-2007 01:02 AM
spider ignores links	Maarten Wijnen	Troubleshooting	2	03-17-2005 03:23 PM
Spider External links to a depth of 1 (1.8.3)	kenazo	How-to Forum	0	10-20-2004 07:28 AM
Spider site with links	dell_10	External Binaries	10	09-20-2004 07:42 AM
no spider my file links	lolodev	Troubleshooting	21	07-16-2004 07:31 PM