PhpDig Version 1.8.8 RC1 Released

[Charter]

Hi. PhpDig version 1.8.8 RC1 has been released as a 'minor+++++' release; it probably should be considered a major release. The changes can be found in the Changelog file.

Previous requirements (before v.1.8.8.RC1):

- PHP safe_mode off, PHP allow_url_fopen on
- See section 3.1 of the v.1.8.7 documentation

New requirements (for v.1.8.8.RC1):

- Previous requirements are still required (oh yeah!)
- MySQL 4.1.7+ with UTF-8 and ability for SET queries
- PHP 4.3.10+ with mbstring, mbstr-enc-trans, mbregex
- Apache with Linux 2.4.26+ and htaccess file ability
- Understanding of iconv to convert files and tables

A fresh install works the same as before, assuming you meet the new requirements. Upgrade instructions (from v.1.8.7 to v.1.8.8 RC1) can be found in the UPGRADE.txt document included in the v.1.8.8. RC1 package. Make a backup of files/tables before attempting an upgrade!

Do check this thread on a regular basis for any possible code changes to PhpDig v.1.8.8 RC1.

[Charter]

This is an improvement to PhpDig v.1.8.8 RC1 for locating and highlighting search terms contained in table content.

In search_functions.php, find the phpdigGetExtracts function.

In this function find:

$where_to_start = mb_strlen($extract_content);

And replace with:

$max_off = (int) min(mb_strlen($query_string)+1,floor(0.80*mb_strlen($extract_content)));
$where_to_start = mb_strlen($extract_content) - $max_off;

Also, in this function find:

$first_bold_spot = max($first_bold_spot - round((SNIPPET_DISPLAY_LENGTH / 2),0), 0);

And replace with:

$first_bold_spot = (int) max($first_bold_spot - round((SNIPPET_DISPLAY_LENGTH / 2),0), 0);

If you downloaded PhpDig v.1.8.8 RC1 after the date of this post, the changes are already included in the package.

[Charter]

This is an improvement to PhpDig v.1.8.8 RC1 handling of Japanese full/half-width characters (thanks to Edomondo).

In config.php find:

/***********************************************************************************************************************/
//---------PATH SETTINGS

And replace with:

/***********************************************************************************************************************/
//---------CONVERT JAPANESE KANA (only for Japanese)

define('ENABLE_JPKANA',false);                   // activates/deactivates japanese kana conversion
define('CONVERT_JPKANA','KVa');                  // see http://us2.php.net/manual/en/function.mb-convert-kana.php for options

/***********************************************************************************************************************/
//---------PATH SETTINGS

In robot_functions.php find the phpdigMakeUTF8 function.

In this function find:

if ($no_convert == 0) {
  return $string;
}
else {
  return 0;
}

And replace with:

if ($no_convert == 0) {
  if (ENABLE_JPKANA == true) {
    $string = @mb_convert_kana($string,CONVERT_JPKANA,"UTF-8");
  }
  return $string;
}
else {
  return 0;
}

In search_functions.php find:

if (!$option) {
     $option = SEARCH_DEFAULT_MODE;
}
if (!in_array($option,array('start','any','exact'))) {
     return 0;
}

if ($query_string) {
    $query_string = mb_eregi_replace("[?]*","",$query_string);
}

if ($query_string) {
    $query_string = phpdigVerifyUTF8($query_string);
}

And replace with:

if ($query_string) {
    if (ENABLE_JPKANA == true) {
      $query_string = @mb_convert_kana($query_string,CONVERT_JPKANA,"UTF-8");
    }
    $query_string = phpdigVerifyUTF8($query_string);
}

Also in search_functions.php find:

$query_to_parse = mb_ereg_replace('&[^ ]+','',$query_to_parse); // avoid &[chars] in the query

And afterwards add:

$query_to_parse = mb_ereg_replace('[?]*','',$query_to_parse); // avoid ? in the query

If you downloaded PhpDig v.1.8.8 RC1 after the date of this post, the changes are already included in the package.

[Charter]

This is an improvement to PhpDig v.1.8.8 RC1 for keeping (quasi)duplicates out of the engine table.

In robot_functions.php find:

  $key = mb_ereg_replace("^([\x00-\x1f]|[\x21-\x2f]|[\x3a-\x40]|[\x5b-\x60]|[\x7b-\x7f])+","",$key); //off front only
  $key = mb_ereg_replace("([\x00-\x1f]|[\x21-\x2f]|[\x3a-\x40]|[\x5b-\x60]|[\x7b-\x7f])+$","",$key); //off back only

And delete these two lines.

Also, in robot_functions.php find:

  for ($token = strtok($text2, $separators); $token !== FALSE; $token = strtok($separators)) {
        if (!isset($nbre_mots[$token]))
            { $nbre_mots[$token] = 1; }
        else
            { $nbre_mots[$token]++; }
       $total++;
  }

And replace with:

  for ($token = strtok($text2, $separators); $token !== FALSE; $token = strtok($separators)) {
        $token = mb_ereg_replace("^([\x00-\x1f]|[\x21-\x2f]|[\x3a-\x40]|[\x5b-\x60]|[\x7b-\x7f])+","",$token); //off front only
        $token = mb_ereg_replace("([\x00-\x1f]|[\x21-\x2f]|[\x3a-\x40]|[\x5b-\x60]|[\x7b-\x7f])+$","",$token); //off back only
        $token = mb_strtolower(trim($token));
        if (mb_strlen($token) > 0) {
          if (!isset($nbre_mots[$token]))
              { $nbre_mots[$token] = 1; }
          else
              { $nbre_mots[$token]++; }
          $total++;
        }
  }

Now run the following queries, adding your table prefix to engine and engine2 if needed:

CREATE TABLE engine2 (
   spider_id mediumint(9) DEFAULT '0' NOT NULL,
   key_id mediumint(9) DEFAULT '0' NOT NULL,
   weight smallint(4) DEFAULT '0' NOT NULL,
   KEY key_id (key_id)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE utf8_general_ci;

INSERT INTO engine2 SELECT spider_id,key_id,sum(weight) AS weight FROM engine GROUP BY spider_id,key_id;

DELETE FROM engine;

INSERT INTO engine SELECT spider_id,key_id,weight FROM engine2;

DROP TABLE engine2;

If you downloaded PhpDig v.1.8.8 RC1 after the date of this post, the code changes are already included in the package.