Problem with indexing "links found : 0"

[IAMHHawaii]

I was having a problem with the spider not being able to index my site,
it would come up with a similare result of

Spidering in progress...

--------------------------------------------------------------------------------
SITE : http://somesite.tld/
Exclude paths :
- @NONE@
No link in temporary table

--------------------------------------------------------------------------------

links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------

after a few days of searching the forums without luck, I found a post the tiped me off to the problem.

I am tesing this search engine out on our dev site which currently has the apache .htaccess password protenction on it to keep it out of public view. I temporarily disabled the protection and this seems to resolve the problem.

If anyone else out there has a better way to do this I would apreciate some input. Ideally I would like to setup the phpDig with authentication access, If anyone knows if this is possible, please let me know how.

[IAMHHawaii]

funny now that I decided to write a post on this I see that there have been others like it in the past, this one for example.

basically a way to get around the authentication withought disableing it is to enter:

username:password@domain.tld as the site instead of just domain.tld