Ban features

[jmitchell]

what if you are indexing other sites?

[rAdoN]

use admin update - "Click on the noway sign to exclude from future indexings" - "Click on the cross to delete the branch" - "Click on the cross to delete a document" - that delete for links indexed not wanted - use FORBIDDEN_EXTENSIONS to prevent for sites - run the cleans - index

ps - no listen

[Slider]

Hello rAdoN,

I apoligize for being such a pain.
You really know your stuff and I will never doubt what I hear from you again.
Thank you so much for being here. Maybe I can return the favor in some way in the future.

[Slider]

I added this line to the config:

define('FORBIDDEN_PATH','(guestbook|forum|cgi-bin|webring|affiliates|links|webrings|banners)');

I added this code to spider.php (the part in bold red is the addition)

//test content-type of this page if not excluded
                           $result_test_http = '';
                           if (!phpdigReadRobots($exclude,$temp_path) && !eregi(FORBIDDEN_EXTENSIONS,$temp_file) && !eregi(FORBIDDEN_PATH,$temp_path)) {
                                $result_test_http = phpdigTestUrl($url_indexing,'date',$cookies);
                           }

I tried the code you gave and even tried variations of it and never was able to get it to ignore a path or directory. This code should be added to the next phpdig version. it's a neccessity if you want to have a little more control over the content that is being indexed and reduce the MySql database.

[rAdoN]

hoorah - instead use book.cgi you make mod - good for path - i mod your mod

PHP Code:

//test content-type of this page if not excluded
$result_test_http = '';
if (!phpdigReadRobots($exclude,$temp_path.$temp_file) && !eregi(FORBIDDEN_EXTENSIONS,$temp_path.$temp_file)) {
     $result_test_http = phpdigTestUrl($url_indexing,'date',$cookies);
} 


[Slider]

I'm not familiar with the book.cgi you were talking about.
The new code you posted would have made it work for the path and filename Congrats!

Thank you very much