Command line vs. admin indexing

wx3 · 08-13-2004, 09:57 AM

Hi all,

I've installed phpdig, and it seems to work fine, except that I can't get it to index regularly via cron or command line.

If I login through the admin interface and click on the green checkbox, phpdig spends about 2-3 minutes indexing the site. When it's done, all the changes to the site are reflected in the search.

If I run the spider from the command line:

Code:

/usr/local/bin/php -f /usr/www/users/username/search/admin/spider.php forceall

I get this:
39258: old priority 0, new priority 18

Spidering in progress...
-----------------------------
SITE : http://www.mywebsite.com/
Exclude paths :
- @NONE@
No link in temporary table
links found : 0
Optimizing tables...
Indexing complete !

What am I doing wrong?

Thanks much,
Kevin

vinyl-junkie · 08-13-2004, 05:41 PM

Welcome to the forum, wx3.

Try this instead:

Code:

 #php -f [PHPDIG_DIR]/admin/spider.php http://host.mydomain.com

Substitute your directory name for [PHPDIG_DIR], and of course your domain name, too.

Let us know how that works for you.

Charter · 08-15-2004, 03:28 PM

Hi, sounds like maybe you need to add the full path to the if statement in the config file?

wx3 · 08-16-2004, 09:44 AM

Hi,

Thanks to both of you for the suggestions. I'm pretty sure the path is set correctly in that first IF statement in the config file (the one that checks to see if relative_script_path is valid). I tried adding the URL explicitly like vinyl suggested, and this is the result:

Spidering in progress...
-----------------------------
SITE : http://www.mywebsite.com/
Exclude paths :
- @NONE@
1:http://www.mywebsite.com/
(time : 00:00:05)
No link in temporary table
links found : 1
Optimizing tables...
Indexing complete !

It found 1 link instead of 0, but still failed to index the site. Any other suggestions?

Thanks,
Kevin

Charter · 08-17-2004, 03:00 PM

What search depth and links per did you use? What happens if you now go update the site?

mirdin · 09-01-2004, 07:13 AM

How do you set the depth and linksper from the command line?

Code:

 #php -f [PHPDIG_DIR]/admin/spider.php http://host.mydomain.com

If this is not possible where can I set the default depth and perlinks in config.php?

Spider · 09-06-2004, 07:03 AM

I seem to have the same problem wx3 has. When I go from command line with:

Code:

/usr/bin/php -f /home/sites/www.vematex.nl/web/phpdig/admin/spider.php forceall

I get:

Code:

<b>Warning</b>:  Undefined variable:  argv in <b>/home/sites/site6/web/phpdig/admin/spider.php</b> on line <b>36</b><br>
<br>
<b>Warning</b>:  Undefined variable:  argc in <b>/home/sites/site6/web/phpdig/admin/spider.php</b> on line <b>49</b><br>
12114: old priority 0, new priority 18
Bezig met spideren...
-----------------------------
SITE : http://www.vematex.nl/.....etc

I use phpdig 1.8.3

Charter · 09-07-2004, 04:07 PM

See around line 25 of the spider.php file.

Spider · 09-08-2004, 12:31 AM

Hi,

I turned on the register_argc_argv on our server. No more 'Undefined variable:'.

Instead of 'forceall' I put 2 'spider'lines in the crontab, each pointing to a different dir. That works.

For now I'm happy. Thanks!

08-13-2004, 09:57 AM	#1
wx3 Green Mole Join Date: Aug 2004 Posts: 3	Command line vs. admin indexing Hi all, I've installed phpdig, and it seems to work fine, except that I can't get it to index regularly via cron or command line. If I login through the admin interface and click on the green checkbox, phpdig spends about 2-3 minutes indexing the site. When it's done, all the changes to the site are reflected in the search. If I run the spider from the command line: Code: /usr/local/bin/php -f /usr/www/users/username/search/admin/spider.php forceall I get this: 39258: old priority 0, new priority 18 Spidering in progress... ----------------------------- SITE : http://www.mywebsite.com/ Exclude paths : - @NONE@ No link in temporary table links found : 0 Optimizing tables... Indexing complete ! What am I doing wrong? Thanks much, Kevin

08-13-2004, 05:41 PM	#2
vinyl-junkie Purple Mole Join Date: Jan 2004 Posts: 694	Welcome to the forum, wx3. Try this instead: Code: #php -f [PHPDIG_DIR]/admin/spider.php http://host.mydomain.com Substitute your directory name for [PHPDIG_DIR], and of course your domain name, too. Let us know how that works for you.

08-15-2004, 03:28 PM	#3
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi, sounds like maybe you need to add the full path to the if statement in the config file? __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

08-17-2004, 03:00 PM	#5
Charter Head Mole Join Date: May 2003 Posts: 2,539	What search depth and links per did you use? What happens if you now go update the site? __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

09-01-2004, 07:13 AM	#6
mirdin Green Mole Join Date: Apr 2004 Posts: 14	How do you set the depth and linksper from the command line? Code: #php -f [PHPDIG_DIR]/admin/spider.php http://host.mydomain.com If this is not possible where can I set the default depth and perlinks in config.php? Last edited by mirdin; 09-01-2004 at 07:18 AM.

08-16-2004, 09:44 AM	#4
wx3 Green Mole Join Date: Aug 2004 Posts: 3	Hi, Thanks to both of you for the suggestions. I'm pretty sure the path is set correctly in that first IF statement in the config file (the one that checks to see if relative_script_path is valid). I tried adding the URL explicitly like vinyl suggested, and this is the result: Spidering in progress... ----------------------------- SITE : http://www.mywebsite.com/ Exclude paths : - @NONE@ 1:http://www.mywebsite.com/ (time : 00:00:05) No link in temporary table links found : 1 Optimizing tables... Indexing complete ! It found 1 link instead of 0, but still failed to index the site. Any other suggestions? Thanks, Kevin

09-07-2004, 04:07 PM	#8
Charter Head Mole Join Date: May 2003 Posts: 2,539	See around line 25 of the spider.php file. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

09-08-2004, 12:31 AM	#9
Spider Green Mole Join Date: Jul 2004 Posts: 13	Hi, I turned on the register_argc_argv on our server. No more 'Undefined variable:'. Instead of 'forceall' I put 2 'spider'lines in the crontab, each pointing to a different dir. That works. For now I'm happy. Thanks!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
command line indexing that actually works	carlaron	Troubleshooting	0	11-06-2006 08:48 PM
problems indexing command line vs. admin panel	obottek	Troubleshooting	1	04-10-2004 03:01 PM
Can I close Putty during command line indexing?	Nosmada	How-to Forum	7	01-23-2004 03:46 PM
Indexing by command line...	Canadian	How-to Forum	4	01-04-2004 06:44 PM
Indexing by command line interface	Skop	Troubleshooting	8	10-14-2003 02:23 AM