View Single Post
Old 04-29-2004, 10:06 AM   #1
-IAN-
Green Mole
 
Join Date: Apr 2004
Posts: 5
Spidering....links found : 0

Hey, I finally got the database created and connected. It was a pain because our Administrator wont allow php to do file uploads, read, or write after we got hacked 3 weeks ago. The install code relies on fopen and writing or creating the file. I finaly got it to work though.
But now when I go to spider our site I just get the following:

Spidering in progress...

--------------------------------------------------------------------------------
SITE : http://clarknexsen/
Exclude paths :
- @NONE@
No link in temporary table

--------------------------------------------------------------------------------

links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------
[Back] to admin interface.

I noticed that in other posts youve suggested changing the robots.txt file to:

User-agent: *
Disallow: /go.php

Now where exactly do I find the "robots.txt" file? Do I need to contect my administrator for it (I access the server remotely)?
thanks!
-IAN- is offline   Reply With Quote