|
08-15-2005, 07:37 AM | #1 |
Green Mole
Join Date: Jul 2005
Posts: 1
|
/~username robots.txt issue - With Fix
PhpDig v.1.8.7
Issue: If a robots.txt file Disallows searching a user directory (/~username) PhpDig interprets this as Disallow: / (everything) Solution: In the robot_functions.php in function phpdigReadRobotsTxt on Line 917 change: Code:
if (eregi('[[:blank:]]*disallow:[[:blank:]]*([/]?([a-z0-9_/*+%.-]*))',$line,$regs)) Code:
if (eregi('[[:blank:]]*disallow:[[:blank:]]*([/]?([a-z0-9_/*+%.-~]*))',$line,$regs)) |
08-15-2005, 09:55 AM | #2 |
Head Mole
Join Date: May 2003
Posts: 2,539
|
Thanks.
Just change it to the following: Code:
[a-z0-9_/*+%.~-] Code:
$allowed_link_chars Code:
[a-z0-9_/*+%.-]* Code:
global $allowed_link_chars;
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension. |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
robots.txt seems to be ignored :? | galacticvoyager | Bug Tracker | 1 | 11-12-2005 12:52 PM |
robots.txt and URL | djavet | How-to Forum | 4 | 01-11-2005 03:19 AM |
robots.txt versus robotsxx.txt | Charter | IPs, SEs, & UAs | 0 | 03-11-2004 06:00 PM |
robots.txt ignored | roy | Troubleshooting | 3 | 02-20-2004 08:02 PM |
robots.txt | renehaentjens | Troubleshooting | 3 | 12-05-2003 02:40 PM |