Indexing .info

christophe · 01-02-2005, 08:16 AM

Hi people
I have a other problem today

My phpDig indexing not the domain with extension ".info"
Perhaps this domain don't exist in USA but in France they have many .info.
How do you indexing .info ?
Please.

vinyl-junkie · 01-02-2005, 10:33 AM

The domain extension shouldn't have anything to do with whether or not phpdig can index the site. It's more likely that the site blocks all but certain spiders, or that individual pages have robots exclusion in place.

What is the site you're trying to index?

christophe · 01-02-2005, 10:42 AM

it's that

http://www.aujardin.info/

vinyl-junkie · 01-02-2005, 10:58 AM

I was able to spider 3 pages on my test server before I stopped the indexing process. However, spidering this site was extremely slow.

What happened when you tried to spider it? Can you post your spider log?

christophe · 01-02-2005, 12:19 PM

I indexed 1 only page.
I don't understand sometimes PhpDig indexing 100 pages et sometimes 1 or 2.
It's strange but i play with

vinyl-junkie · 01-02-2005, 12:28 PM

What values did you set for "links per" and "search depth?" Remember what the admin page says:

Quote:

- Search depth of zero tries to crawl just that page regardless of links per
- Set links per depth to the max number of links to check at each depth
- Links per depth of zero means to check for all links at each seach depth

christophe · 01-03-2005, 09:13 AM

I can indexing the domain in .info but with the mod "addurl", the people doesn't can add a .info domain.

Charter · 01-03-2005, 03:12 PM

Sounds like a regex issue. Check with the the mod "addurl" author, edit the mod "addurl" code, or offer to pay someone to edit the mod "addurl" for you.

Dave A · 01-08-2005, 12:52 PM

The spider does index domains with that extension (.info)
I have used it to index quite a few myself so it may be the robots.txt file or htaccess that is blocking you.

christophe · 01-09-2005, 04:03 AM

Ah yes Perhaps (may be)
Thanks

01-02-2005, 08:16 AM	#1
christophe Green Mole Join Date: Dec 2004 Posts: 25	Indexing .info Hi people I have a other problem today My phpDig indexing not the domain with extension ".info" Perhaps this domain don't exist in USA but in France they have many .info. How do you indexing .info ? Please.

01-03-2005, 03:12 PM	#8
Charter Head Mole Join Date: May 2003 Posts: 2,539	Sounds like a regex issue. Check with the the mod "addurl" author, edit the mod "addurl" code, or offer to pay someone to edit the mod "addurl" for you. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

01-08-2005, 12:52 PM	#9
Dave A Purple Mole Join Date: Aug 2004 Location: North Island New Zealand Posts: 170	Re indexing of Info domains. The spider does index domains with that extension (.info) I have used it to index quite a few myself so it may be the robots.txt file or htaccess that is blocking you.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
php includes template info	5wayshost	Coding & Tutorials	0	12-31-2006 09:59 PM
Results to display info from mysql table	kjhay	How-to Forum	4	09-03-2005 01:34 PM
More site info	AllKnightAccess	Mod Requests	0	01-22-2005 01:39 PM
Version info to admin panel	manfred	Mod Requests	1	02-19-2004 01:18 PM
General Info	Charter	Feedback & News	0	09-03-2003 02:25 PM

01-02-2005, 10:33 AM	#2
vinyl-junkie Purple Mole Join Date: Jan 2004 Posts: 694	The domain extension shouldn't have anything to do with whether or not phpdig can index the site. It's more likely that the site blocks all but certain spiders, or that individual pages have robots exclusion in place. What is the site you're trying to index?

01-02-2005, 10:42 AM	#3
christophe Green Mole Join Date: Dec 2004 Posts: 25	it's that http://www.aujardin.info/

01-02-2005, 10:58 AM	#4
vinyl-junkie Purple Mole Join Date: Jan 2004 Posts: 694	I was able to spider 3 pages on my test server before I stopped the indexing process. However, spidering this site was extremely slow. What happened when you tried to spider it? Can you post your spider log?

01-02-2005, 12:19 PM	#5
christophe Green Mole Join Date: Dec 2004 Posts: 25	I indexed 1 only page. I don't understand sometimes PhpDig indexing 100 pages et sometimes 1 or 2. It's strange but i play with

01-03-2005, 09:13 AM	#7
christophe Green Mole Join Date: Dec 2004 Posts: 25	I can indexing the domain in .info but with the mod "addurl", the people doesn't can add a .info domain.

01-09-2005, 04:03 AM	#10
christophe Green Mole Join Date: Dec 2004 Posts: 25	Ah yes Perhaps (may be) Thanks