spider.php via bash

[tomas]

hello together,

i'm trying to start the spider-process via shell - but nothing happens - so i read all articles concerning this in the forum.

but nothing works!

/usr/local/bin/php4 -f /phpdig/admin/spider.php http://www.domain.com >> phpdigg.log

when i take the -w switch(instead of -f) i see the code from spider.php => this tells me all paths are ok
several other scripts run without problems from shell and cron

but spider.php won't :-(

sending the command=>new prompt=>but nothing happens

please help :-)

[Charter]

Hi. Try this and then let me know what's in the phpdigg.log file:

/usr/local/bin/php4 -f /phpdig/admin/spider.php http://www.domain.com > phpdigg.log 2>&1

[tomas]

hello charter,

this is the fastes response i ever got - thanks

content of logfile:
X-Powered-By: PHP/4.2.3
Content-type: text/html

[Charter]

Hi. What does the following produce?

CD to the admin directory and then do the following:

php -f spider.php http://www.domain.com > phpdigg.log 2>&1

[tomas]

hi charter,

great!
no i hope it will run for the next 7 hours and index all sites.


thanks in advance
tomas from germany

[tomas]

but one question remains,

when calling the script with cron i need the full path
from root :-(

[Charter]

Hi. If you are following this tutorial, and you place everything in the admin directory, then you can use the following to spider on the 1st and 15th of every month:

0 0 1,15 * * php -f spider.php cronlist.txt > phpdigg.log

[tomas]

hi charter,

ok - i try this out - but why does it not
work with the full path - i do this with some
other php scripts and they work fine???

[Charter]

Hi. Full path to PHP or other scripts?

0 0 1,15 * * /full/path/to/php -f /full/path/to/admin/spider.php cronlist.txt > phpdigg.log

What OS are you using?

[tomas]

full path to other scripts
debian
won't i loose all other cronjobs in my crontab
doing it like the tutorial?

[Charter]

Hi. The command first posted was the following:

/usr/local/bin/php4 -f /phpdig/admin/spider.php http://www.domain.com >> phpdigg.log

Was /phpdig/admin/spider.php the full path to spider.php? Perhaps try /usr/local/bin/php or just php instead of /usr/local/bin/php4 in the command.

I'm not all that familiar with Debain, but with Linux you can set a number of cronjobs up to the limit imposed by a host. Perhaps try setting a test cronjob that prints some text to a file and see if it removes the PhpDig cronjob.

[tomas]

yes - as i thought this overwrites the existing crontab from the logged user -
but charter - why all this ???
a lot of other scripts called by crontab work fine
why can't i call this from my user directory this way like all the others:
5 * * * 1,2,3,4,5 /usr/local/bin/php4 /homepages/22/d86961/htdocs/cron/rep_01.php > /dev/null

[Charter]

Hi. How many other cronjobs do you have running? Maybe you are at your limit, but I'm guessing not. One thing is that the config.php file has the following code:

PHP Code:

if ((isset($relative_script_path)) && ($relative_script_path != ".") && ($relative_script_path != "..")) {
  exit();
} 


This code was implemented for security reasons. If the $relative_script_path from shell is different than dot or double dot, then the process will exit. If this is the case, just add the relative path needed in the above code.

[tomas]

gotcha -
this is the troublemaker:
doing this:
/*
if ((isset($relative_script_path)) && ($relative_script_path != ".") && ($relative_script_path != "..")) {
exit();
}
if (eregi("config.php",$_SERVER['SCRIPT_FILENAME']) || eregi("config.php",$_SERVER['REQUEST_URI'])) {
exit();
}
*/

now it runs

[Charter]

Hi. Glad it's now working, but you don't want to comment that section out. It was added for security reasons, and unless you are running locally (no other person has access) then that is opening you up to a world of trouble! Rather you need to allow for the relative path in the code, the same as was done for dot and double dot.