Well spotted!
I am indeed using a mac (MacOS 10.3.3 Server), and when I looked at my robots.txt I saw that I had created a .txt file with macintosh line endings, as you suggested.
I've now saved as unix and indexing is working correctly. Superb!
Thanks Charter.