PhpDig.net - Why scan apache multi indexes?

PhpDig.net (http://www.phpdig.net/forum/index.php)

- How-to Forum (http://www.phpdig.net/forum/forumdisplay.php?f=33)

- - Why scan apache multi indexes? (http://www.phpdig.net/forum/showthread.php?t=1053)

Why scan apache multi indexes?

The apache manuals show that you can specify sort orders for automatic directory listing e.g.

http://myserver/mydir/?C=N;O=D

The config.php has options for 16 variations for apache multi indexes. The default is all enabled. I see four of them in use when spidering my site.

1. Why do we need to rescan a directory with different sort orders at all? My server always responds with "Duplicate of an existing document".

2. Why does my spidering only use four of them?

3. Can I safely turn it off?

Thanks
Rob

Hi. The following is in the spider.php file:

PHP Code:


		
			
else if (!isset($apache_indexes[$lien['file']])) {

// check link and if okay include in index

}

so it's trying not to include multiple copies formed from Apache FancyIndexing.