How can I restart spidering after crash?

[yapuka]

Hello,

I started spidering a large mailing list archive Friday, and everything seemed to be going fine, until my browser crashed yesterday, after 3000 pages.

When I relaunched PhpDig, the sites was marked as 'Locked'. Since I didn't want to restart individually all the lines, I changed the value to of 'locked' to '0' in table 'sites'.

But now, PhpDig simply will not spider the site, saying it's been done recently!

How can I start it again?

[Charter]

Hi. To unlock a site from the admin panel, choose the site, click the update button, and click the unlock link. Reindex can be based on meta tag or LIMIT_DAYS as set in the config file. You may find this thread useful. Also for larger sites it might be better to use shell when indexing if available.

[roger]

Hi.

I have the same problem and have to enter the database to remove the '1' lunlock value. The unlock link doesn't work. Can you help me get it working.

Roger

[bloodjelly]

Try emptying the temp spider table, killing the spider process if it's still going, and unlocking the site, then spider again.

[roger]

Hi, how do I

emptying the temp spider table

without going into the database which is what I'm trying to avoid by asking how to get the link to work

[bloodjelly]

In the admin panel, click "delete site" without any site selected, and it will empty the temp spider table.

[roger]

Hi. I don't want to press the issue as I've not slept for two days but after executing the delete 'button' I still have 2 locked sites ???

[bloodjelly]

Hi - no worries. Did you unlock them like Charter said above? Does the temporary table keep filling up after you empty it? If yes to the second question, you're still running a spider process that you should kill (through the shell, with the "kill PID" command)

Sometimes this happens to me, when I unlock a site only to have it keep relocking itself. If I can't kill the process for some reason I usually just keep emptying the tempspider table a few times until PhpDig chokes and it stops filling up. I know that's not the best way to do it but it's worked for me. Good luck.

[roger]

I'm not sure what's going on at the moment except that I can unlocj vis the database. So my thoughts are how to make a or the link work to execute an SQL command that will do it for me.

I imagine that's what's supposed to happen any way.

I've never doen any SQL so it's probably about time I did ??????

Thanks for your support BJ

[bloodjelly]

Sorry Roger I guess I was answering yapuka's question and kind of confusing you. If the link doesn't work, maybe you can tell us exactly what happens. Does it unlock briefly and then lock again? When you unlock it directly through the DB does it stay unlocked? The SQL would be something like this:

"UPDATE sites SET locked='0' WHERE site_url = 'http://www.searched_website.com';"

But if that worked, the unlock link in the admin panel should work as well. Hmm...maybe someone else can throw out some ideas.

[roger]

Hi. Thanks

When I do a direct chenge via 'phpmyadmin' and just change the 1 to 0 it unlocks and there's no problem, it stays unlocked.

Nothing else works. The <update> <unlock> does nothing nor does <deleteing> with no site highlighted.

It shouldn't be a big problem as it only crashes, so far, when I try to quit a spider as I can see it's doing a never-ending forum with even more 'sessions' or some other reason - then it just locks the site and that's it. maybe the spidering it just going on - somehere - somehow ?????

Excuse my ignorance but how do I impliment the SQL code via an html link? If I can get that to work I'll then look at why the <unlock> link doesn't do the same. I will have a look at the current code once I've found the page.

Thanks again for your help

[roger]

Hi Again.

My new thoughts are as follows:

As I couldn't initiate a database when I tried to personally install Version 1.8 I resorted to having an automatic insatall of version 1.6 provide by my webhost's add-on.

Then i found that the <update> link didn't work and found that there was no <update.php> file so I ftp'd across the one in version 1.8 from my local machine.

So maybe either there more wrong with my 1.6 version or the 1.8 <update.php> is incompatible with v1.6.

Maybe I should try and transfer all the other 1.8 files ???

[roger]

Oops! I've transfered all the new files v1.8 except configure.php and connect.php and now I get no results although the database seems intact ???