|
01-21-2008, 02:16 AM | #1 |
Green Mole
Join Date: Jan 2008
Posts: 1
|
Selective Indexing of URL Containing a <keyword>
Hi Forum,
I'm trying to customize phpDig to do the following: - I want to spider all links in a particular domain. - Only specific URL's should be indexed, especially URL's containing the keyword - <rest of URL>"showproperty"<rest of URL>. - My first attempt was to modify the "phpdigRewriteURL" function inside the "robot_functions.php" API, in the following way: Code: if (!eregi("showproperty", $eval)) { return -1; } so that any URL containing the keyword is not indexed. This, however, results in all links not being followed, and only links containing the keyword... - My second attempt involves playing with the "$ok_for_index" parameter in the "spider.php" API: //let's go for indexing the content if ($ok_for_index == 1) { ... so that the "$ok_for_index" is only set to 1 if the URL being indexed contains the "showproperty" keyword. My question is; am I going in the correct direction with this, or is there a more optimal approach? Thanks, Leith |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
<!-- phpdigInclude --> and <!-- phpdigExclude --> doesn`t work | Paka76 | How-to Forum | 0 | 12-06-2005 05:44 AM |
Indexing "<word>-<word>"? | FaberFedor | How-to Forum | 23 | 02-28-2005 03:35 AM |
Where have all the <titles> gone? | siliconkibou | Troubleshooting | 8 | 08-27-2004 07:09 AM |
Instructions for use <!-- phpdigExclude --> and <!-- phpdigInclude --> | maquido | How-to Forum | 1 | 06-02-2004 03:36 AM |
< phpdigInclude > | oliviert | Troubleshooting | 12 | 05-19-2004 02:13 AM |