Spidering problems

Dave A · 07-20-2005, 04:14 AM

I can't stop any spidering going on, if I go to phpadmin and do a search for locked and place a 1 so anything that has a 1 is locked I get two websites shown as locked, if I press the refresh button I get a different two or sometimes three websites showing up as spidering and locked.
I can't seem to stop a website from starting to spider all on it's own
Even if I go down the list of web sites on the admin pannel they often show up as okay and not being spidered but as soon as I click clean common words it shows an error nessage stating that it can't be done when websites ar locked for spidering.
So I need to find some way to stop this, I have reset the server but it started again, I have gone into phpmy admin and repaired all the databases but I still can't get PHPDIG to stop spidering.
If any one has an idea I can try please can they get back to me..

Many regards
Everyone
Dave A

Charter · 07-23-2005, 12:36 PM

Do you see content in the tempspider table when a site is locked? If you click clean common words, the sites will also lock while the cleaning is done.

Dave A · 07-23-2005, 01:54 PM

Hi Charter I can't use the clean common words function becuase it always reports back that it can't do this because a site is locked.
I have gone into phpmy admin and tried to set all sites as unlocked where locked =0 but as soon as i do , then a search for locked files always show one or two locked, each time I refresh the results it shows different files as locked.

The results are mostly showing 100% but with no content when a search is done.

Your assistance is needed my friend

Dave A · 07-23-2005, 01:55 PM

The temp folder never shows any content when this is happening

Charter · 07-23-2005, 02:13 PM

In the sites table, try setting all the values in both the locked and stopped columns to zero.

Dave A · 07-23-2005, 03:19 PM

I have tried to set all the tables to unlocked but as soon as you go back to the menu to try and clean the common words list , it reports that i't can't because sites are locked.

I have tried to unlock them all but if I go to phpmyadmin and search for locked=1 then two are threee sites are locked, if I then edit that site the locked part of the dayabase reports it as ZERO?

Charter · 07-23-2005, 06:00 PM

After you run the following query, and then go back to the PhpDig admin panel, there is still a locked site? Do you have any PhpDig related cron job running?

Code:

UPDATE sites SET locked = 0, stopped = 0;

Dave A · 08-20-2005, 07:52 PM

Just to let you know that it is functioning okay, it has the following results:
DataBase status
Hosts : 9434 Entries
Pages : 149481 Entries
Index : 29167923 Entries
Keywords : 1516635 Entries
Temporary table : 0 Entries

This must be pushing it to the boundaries a little

lhukaz · 08-21-2005, 08:46 AM

Dave A

Do you have menssenger??
If you have add me! -> lhpsobral@msn.com

Thank's

07-20-2005, 04:14 AM	#1
Dave A Purple Mole Join Date: Aug 2004 Location: North Island New Zealand Posts: 170	Spidering problems I can't stop any spidering going on, if I go to phpadmin and do a search for locked and place a 1 so anything that has a 1 is locked I get two websites shown as locked, if I press the refresh button I get a different two or sometimes three websites showing up as spidering and locked. I can't seem to stop a website from starting to spider all on it's own Even if I go down the list of web sites on the admin pannel they often show up as okay and not being spidered but as soon as I click clean common words it shows an error nessage stating that it can't be done when websites ar locked for spidering. So I need to find some way to stop this, I have reset the server but it started again, I have gone into phpmy admin and repaired all the databases but I still can't get PHPDIG to stop spidering. If any one has an idea I can try please can they get back to me.. Many regards Everyone Dave A

07-23-2005, 12:36 PM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Do you see content in the tempspider table when a site is locked? If you click clean common words, the sites will also lock while the cleaning is done. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

07-23-2005, 01:54 PM	#3
Dave A Purple Mole Join Date: Aug 2004 Location: North Island New Zealand Posts: 170	re spider problem Hi Charter I can't use the clean common words function becuase it always reports back that it can't do this because a site is locked. I have gone into phpmy admin and tried to set all sites as unlocked where locked =0 but as soon as i do , then a search for locked files always show one or two locked, each time I refresh the results it shows different files as locked. The results are mostly showing 100% but with no content when a search is done. Your assistance is needed my friend

07-23-2005, 02:13 PM	#5
Charter Head Mole Join Date: May 2003 Posts: 2,539	In the sites table, try setting all the values in both the locked and stopped columns to zero. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

07-23-2005, 06:00 PM	#7
Charter Head Mole Join Date: May 2003 Posts: 2,539	After you run the following query, and then go back to the PhpDig admin panel, there is still a locked site? Do you have any PhpDig related cron job running? Code: UPDATE sites SET locked = 0, stopped = 0; __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

07-23-2005, 01:55 PM	#4
Dave A Purple Mole Join Date: Aug 2004 Location: North Island New Zealand Posts: 170	The temp folder never shows any content when this is happening

07-23-2005, 03:19 PM	#6
Dave A Purple Mole Join Date: Aug 2004 Location: North Island New Zealand Posts: 170	I have tried to set all the tables to unlocked but as soon as you go back to the menu to try and clean the common words list , it reports that i't can't because sites are locked. I have tried to unlock them all but if I go to phpmyadmin and search for locked=1 then two are threee sites are locked, if I then edit that site the locked part of the dayabase reports it as ZERO?

08-20-2005, 07:52 PM	#8
Dave A Purple Mole Join Date: Aug 2004 Location: North Island New Zealand Posts: 170	Just to let you know that it is functioning okay, it has the following results: DataBase status Hosts : 9434 Entries Pages : 149481 Entries Index : 29167923 Entries Keywords : 1516635 Entries Temporary table : 0 Entries This must be pushing it to the boundaries a little

08-21-2005, 08:46 AM	#9
lhukaz Green Mole Join Date: Aug 2005 Posts: 2	Dave A Do you have menssenger?? If you have add me! -> lhpsobral@msn.com Thank's

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Mod-rewrite = spidering / URL problems	jcnorris	Troubleshooting	1	10-26-2006 10:38 AM
spidering in domain 2 problems	dhorwitz	Bug Tracker	2	05-18-2005 05:01 AM
Spidering Problems on a Windows Server Website	vinyl-junkie	Troubleshooting	23	02-20-2004 07:44 PM
Problems spidering dynamic site	Ph0nK	Troubleshooting	1	01-13-2004 04:39 PM