Indexing always stops with locked site

jamie · 01-05-2004, 01:37 PM

Hello everybody,

I've installed PhpDig onto a server which is not mine.

Search runs ok, but indexing seems a bit difficult: whenever I start indexing my page with level 1, the browser stops and does nothing more (after not even a minute). When I go back to the admin interface, I see that the site is locked. I can redo the process after unlocking the site, but everytime it only takes a couple of links more - but never all.

As you can see here (http://iifs.de/phpdig/phpinfo.php,) the safe_mode is Off, but I don't know how I could change that (as it's not my server). I've even set the PHPDIG_DEFAULT_INDEX on false, but nothing changed.

Can someone please help me? Thanks.

MonoNexo · 01-05-2004, 06:03 PM

Try letting it run for an extended period of time, more than a minute. Sometimes, if it gets a large page, it will slow down considerably, or do chunks of the site. I have a fairly large (170+) page site, and for a few pages, it will decrease in speed incredibly, but if I leave it be, it will continue to work.

With the site being locked, that means that it is still working at spidering the site. Just give it time, it'll get through. Just a bit slow perhaps

.

jamie · 01-06-2004, 09:31 AM

Hello MonoNexo,
I've let it run the whole day, but it hasn't spidered more than 7 pages and the site was locked after the first minute.

Any more ideas?

Charter · 01-06-2004, 11:29 AM

Hi. A site is locked during the spidering process. This does not mean anything bad. It just means that PhpDig is working on that site. If spidering is prematurely interupted, a site can remain locked. In that case, just go to the admin panel, select the site, click the update button, and then click the unlock link.

If you are spidering a site that has the same links over and over, then PhpDig will crawl and determine that such content is duplicated. You might try utilizing the PhpDig exclude/include comments to avoid this. Also, MonoNexo's post is good. Try indexing the site in parts rather than selecting a high search depth.

One other thing, as you are using the browser interface for PhpDig, once you start an index, do not use that browser window for other things until the index is completed. You might want to try indexing by shell instead, as this bypasses the browser interface.

01-05-2004, 01:37 PM	#1
jamie Green Mole Join Date: Jan 2004 Posts: 2	Indexing always stops with locked site Hello everybody, I've installed PhpDig onto a server which is not mine. Search runs ok, but indexing seems a bit difficult: whenever I start indexing my page with level 1, the browser stops and does nothing more (after not even a minute). When I go back to the admin interface, I see that the site is locked. I can redo the process after unlocking the site, but everytime it only takes a couple of links more - but never all. As you can see here (http://iifs.de/phpdig/phpinfo.php,) the safe_mode is Off, but I don't know how I could change that (as it's not my server). I've even set the PHPDIG_DEFAULT_INDEX on false, but nothing changed. Can someone please help me? Thanks.

01-06-2004, 11:29 AM	#4
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. A site is locked during the spidering process. This does not mean anything bad. It just means that PhpDig is working on that site. If spidering is prematurely interupted, a site can remain locked. In that case, just go to the admin panel, select the site, click the update button, and then click the unlock link. If you are spidering a site that has the same links over and over, then PhpDig will crawl and determine that such content is duplicated. You might try utilizing the PhpDig exclude/include comments to avoid this. Also, MonoNexo's post is good. Try indexing the site in parts rather than selecting a high search depth. One other thing, as you are using the browser interface for PhpDig, once you start an index, do not use that browser window for other things until the index is completed. You might want to try indexing by shell instead, as this bypasses the browser interface. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Imcomplete spider : locked site but nothing append	beausej	Troubleshooting	1	07-18-2005 04:48 PM
Spidering Stops, Site is Locked after 7 Pages	nevsie	Troubleshooting	3	03-02-2005 04:52 PM
Locked Site	olivier	Troubleshooting	3	02-11-2005 07:21 AM
Indexing stops after 2 seconds	msberryuk	Troubleshooting	3	01-13-2005 06:28 AM
locked site and commandlineinterface	baskamer	How-to Forum	1	02-05-2004 09:03 AM

01-05-2004, 06:03 PM	#2
MonoNexo Green Mole Join Date: Jan 2004 Posts: 15	Try letting it run for an extended period of time, more than a minute. Sometimes, if it gets a large page, it will slow down considerably, or do chunks of the site. I have a fairly large (170+) page site, and for a few pages, it will decrease in speed incredibly, but if I leave it be, it will continue to work. With the site being locked, that means that it is still working at spidering the site. Just give it time, it'll get through. Just a bit slow perhaps .

01-06-2004, 09:31 AM	#3
jamie Green Mole Join Date: Jan 2004 Posts: 2	Hello MonoNexo, I've let it run the whole day, but it hasn't spidered more than 7 pages and the site was locked after the first minute. Any more ideas?