Fixing spider.php, protecting from locking site after timeout or users stop

[Konstantine]

I found, that if your PHP has time out (for example 30 secs) and spider have no time to complete indexing, your site will be locked and you'll have to go back to admin and unlock it. It's not good, so next code fix it.

Add this code into spider.php file at the begining (from 12 line, after comments, for example):

PHP Code:

function Unlock()
{
  global $site_id, $id_connect, $run_mode, $mode;

  // unlock site if not normal terminate
if(connection_status())
{
  mysql_query('UPDATE '.PHPDIG_DB_PREFIX.'sites SET locked=0 WHERE site_id='.$site_id,$id_connect);
  print "<br />Site was unlocked<br />";

  if ($run_mode == 'http')
  { ?>
<hr />
<a href="index.php" >[<?php phpdigPrnMsg('back') ?>]</a> <?php phpdigPrnMsg('to_admin') ?>.
<?php
    if (isset($mode) && isset($site_id) && $mode == 'small') {
      print '<br /><a href="update_frame.php?site_id='.$site_id.'" >['.phpdigMsg('back').']</a> '.phpdigMsg('to_update').'.';
    }
  } else {
      print $br;
  }
}

return;
}

register_shutdown_function("Unlock");

Hope, it will be in next versions and will help you!!!

[Konstantine]

PHP Code:

function Unlock()
{
  global $site_id, $id_connect, $run_mode, $mode, $br;

  // unlock site if not normal terminate
if(connection_status())
{
  mysql_query('UPDATE '.PHPDIG_DB_PREFIX.'sites SET locked=0 WHERE site_id='.$site_id,$id_connect);
  print $br."Site was unlocked".$br;

  if ($run_mode == 'http')
  { ?>
<hr />
<a href="index.php" >[<?php phpdigPrnMsg('back') ?>]</a> <?php phpdigPrnMsg('to_admin') ?>.
<?php
    if (isset($mode) && isset($site_id) && $mode == 'small') {
      print '<br /><a href="update_frame.php?site_id='.$site_id.'" >['.phpdigMsg('back').']</a> '.phpdigMsg('to_update').'.';
    }
  } else {
      print $br;
  }
}

return;
}

register_shutdown_function("Unlock");

[Konstantine]

If we change index.php file, method of posting from POST to GET

PHP Code:

<form class="grey" action="spider.php" method="get" 


And changing function Unlock (see above) to that code:

PHP Code:

function Unlock()
{
  global $site_id, $id_connect, $run_mode, $mode, $br;

  // unlock site if not normal terminate
if(connection_status())
{
  // clean the tempspider table
  $query = "DELETE FROM ".PHPDIG_DB_PREFIX."tempspider WHERE site_id=$site_id AND (error = 1 OR indexed = 1)";
  mysql_query($query,$id_connect);
  mysql_query('UPDATE '.PHPDIG_DB_PREFIX.'sites SET locked=0 WHERE site_id='.$site_id,$id_connect);
  print $br."Site was unlocked".$br;

  if ($run_mode == 'http')
  { ?>
<hr />
<a href="index.php" >[<?php phpdigPrnMsg('back') ?>]</a> <?php phpdigPrnMsg('to_admin') ?>.
<?php
    if (isset($mode) && isset($site_id) && $mode == 'small') {
      print '<br /><a href="update_frame.php?site_id='.$site_id.'" >['.phpdigMsg('back').']</a> '.phpdigMsg('to_update').'.';
    }

    print <<< HTML
<script>
<!--
window.location.reload();
//-->
</script>
HTML;
  } else {
      print $br;
  }

}

return;
}

Have a nice day!

[miccheng]

Was this implemented for version 1.8?

I have problems with indexing one of my client's website. I have no problems spidering on my own server. Have installed this script for my installation of PHPdig. Will see if it solves the porblem.

Thanks.