PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > How-to Forum

Reply
 
Thread Tools
Old 01-21-2008, 02:16 AM   #1
Leith
Green Mole
 
Join Date: Jan 2008
Posts: 1
Selective Indexing of URL Containing a <keyword>

Hi Forum,

I'm trying to customize phpDig to do the following:

- I want to spider all links in a particular domain.

- Only specific URL's should be indexed, especially URL's containing the keyword - <rest of URL>"showproperty"<rest of URL>.

- My first attempt was to modify the "phpdigRewriteURL" function inside the "robot_functions.php" API, in the following way:
Code:
if (!eregi("showproperty", $eval)) {
return -1;
}
so that any URL containing the keyword is not indexed. This, however, results in all links not being followed, and only links containing the keyword...

- My second attempt involves playing with the "$ok_for_index" parameter in the "spider.php" API:
//let's go for indexing the content
if ($ok_for_index == 1) {
...
so that the "$ok_for_index" is only set to 1 if the URL being indexed contains the "showproperty" keyword.

My question is; am I going in the correct direction with this, or is there a more optimal approach?

Thanks,
Leith
Leith is offline   Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
<!-- phpdigInclude --> and <!-- phpdigExclude --> doesn`t work Paka76 How-to Forum 0 12-06-2005 05:44 AM
Indexing "<word>-<word>"? FaberFedor How-to Forum 23 02-28-2005 03:35 AM
Where have all the <titles> gone? siliconkibou Troubleshooting 8 08-27-2004 07:09 AM
Instructions for use <!-- phpdigExclude --> and <!-- phpdigInclude --> maquido How-to Forum 1 06-02-2004 03:36 AM
< phpdigInclude > oliviert Troubleshooting 12 05-19-2004 02:13 AM


All times are GMT -8. The time now is 11:18 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.