|
Download -
PHP Reference -
Links -
Demo 1.8.9 RC1 -
API
License - Credits - Copying - Changelog - Documentation |
|
|
03-11-2004, 07:00 PM
|
#1 |
|
Head Mole
Join Date: May 2003
Posts: 2,539
|
robots.txt versus robotsxx.txt
Hi. It's one thing to not parse a robots.txt file correctly, but it's another thing to ask for a nonstandard filename.
218.232.120.81 - - [10/Mar/2004:22:36:20 -0800] "GET /robotsxx.txt HTTP/1.0" 404 0 "-" "PlantyNet_WebRobot_V1.9 dhkang@plantynet.com" 218.232.120.81 - - [10/Mar/2004:22:36:21 -0800] "GET / HTTP/1.0" 200 37469 "-" "PlantyNet_WebRobot_V1.9 dhkang@plantynet.com"
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension. |
|
|
 |
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| robots.txt seems to be ignored :? | galacticvoyager | Bug Tracker | 1 | 11-12-2005 01:52 PM |
| robots.txt and URL | djavet | How-to Forum | 4 | 01-11-2005 04:19 AM |
| robots.txt comments | edkay | Mod Submissions | 2 | 03-12-2004 01:41 PM |
| robots.txt ignored | roy | Troubleshooting | 3 | 02-20-2004 09:02 PM |
| robots.txt | renehaentjens | Troubleshooting | 3 | 12-05-2003 03:40 PM |
