indexing from command line with text file

Wayne McBryde · 01-08-2004, 06:26 PM

I can reindex from the command line with no problem, but it will only reindex the sites allready in the database.
From shell and in the phpdig-admin folder, I enter:
"php -f spider.php cronlist.txt"
It will reindex the sites allready in the database but not the sites in the text file.
If I enter:
"php -f spider.php http://www.domain.com"
I get the same result.

PHP ver is 4.1.2

I also tried to setup the cron job but it's not working. However I would perfet to use the command line.

Charter · 01-09-2004, 04:54 AM

Hi. When you run php -f spider.php http://www.newdomain.com what output does PhpDig give?

Wayne McBryde · 01-10-2004, 06:04 PM

The atached file (graphic) shows the command and output. When I enter php -f spider.php http://www.150cam.com
If I enter:
php -f spider.php all
or
php -f spider.php text_file.txt
or
php -f spider.php
or
php -f spider.php http://www.150cam.com
The result is the same. It will reindex the sites in the database but thats all.

If the graphic does not show up, it is at lakenormansweb.com/phpdig_index.jpg

Also Charter, I'm sorry I put this post in the wrong form.

Thanks for your help.

Charter · 01-11-2004, 01:47 PM

Hi. Are you able to index that site via the browser interface? A test run via shell gave the following output. Curious. Maybe this is related to the FP issue experienced previously?

Code:

> php -f spider.php http://www.150cam.com
Spidering in progress...
-----------------------------
SITE : http://www.150cam.com/
Exclude paths :
- @NONE@
+1:http://www.150cam.com/
(time : 00:00:03)
+
level 1...
XDuplicate of an existing document
2:http://www.150cam.com/index.html
(time : 00:00:05)

No link in temporary table
links found : 2
Optimizing tables...
Indexing complete !

Don't worry about posting in the wrong forum. I've moved things around to differentiate between troubleshooting and how-tos.

Wayne McBryde · 01-11-2004, 05:52 PM

No problems from the browser interface. This site and many more spidered on my main site. I have phpdig installed on 2 servers, 1 for production and 1 for testing. Spidering from the command line has the same problem on both. Both also have Front Page installed. I'll try to setup a site w/o Front Page and see what happens. Any ideas about what to try if Front Page does apear to be the problem?

Wayne McBryde · 01-11-2004, 07:27 PM

I setup a new site with a new domain name. I did not install Front Page. It has the same problem as the site with Front Page installed.

Charter · 01-11-2004, 07:31 PM

Hi. Are you getting any errors in your PHP logs when you try and index via shell? Also, can you post a link to your PHP info?

Charter · 01-11-2004, 08:35 PM

Hi. Try setting register_argc_argv to On in the php.ini file.

Wayne McBryde · 01-12-2004, 06:56 PM

Charter,

Changing register_argc_argv to On in the php.ini file did the trick.

IT WORKS GREAT NOW!! THANKS

01-08-2004, 06:26 PM	#1
Wayne McBryde Orange Mole Join Date: Oct 2003 Location: NC, USA Posts: 34	indexing from command line with text file I can reindex from the command line with no problem, but it will only reindex the sites allready in the database. From shell and in the phpdig-admin folder, I enter: "php -f spider.php cronlist.txt" It will reindex the sites allready in the database but not the sites in the text file. If I enter: "php -f spider.php http://www.domain.com" I get the same result. PHP ver is 4.1.2 I also tried to setup the cron job but it's not working. However I would perfet to use the command line. __________________ Wayne Mcbryde http://LakeNormansWeb.com We search all of Lake Norman!

01-09-2004, 04:54 AM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. When you run php -f spider.php http://www.newdomain.com what output does PhpDig give? __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

01-11-2004, 01:47 PM	#4
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Are you able to index that site via the browser interface? A test run via shell gave the following output. Curious. Maybe this is related to the FP issue experienced previously? Code: > php -f spider.php http://www.150cam.com Spidering in progress... ----------------------------- SITE : http://www.150cam.com/ Exclude paths : - @NONE@ +1:http://www.150cam.com/ (time : 00:00:03) + level 1... XDuplicate of an existing document 2:http://www.150cam.com/index.html (time : 00:00:05) No link in temporary table links found : 2 Optimizing tables... Indexing complete ! Don't worry about posting in the wrong forum. I've moved things around to differentiate between troubleshooting and how-tos. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

01-11-2004, 05:52 PM	#5
Wayne McBryde Orange Mole Join Date: Oct 2003 Location: NC, USA Posts: 34	No problems from the browser interface. This site and many more spidered on my main site. I have phpdig installed on 2 servers, 1 for production and 1 for testing. Spidering from the command line has the same problem on both. Both also have Front Page installed. I'll try to setup a site w/o Front Page and see what happens. Any ideas about what to try if Front Page does apear to be the problem? __________________ Wayne Mcbryde http://LakeNormansWeb.com We search all of Lake Norman!

01-11-2004, 07:27 PM	#6
Wayne McBryde Orange Mole Join Date: Oct 2003 Location: NC, USA Posts: 34	I setup a new site with a new domain name. I did not install Front Page. It has the same problem as the site with Front Page installed. __________________ Wayne Mcbryde http://LakeNormansWeb.com We search all of Lake Norman!

01-11-2004, 07:31 PM	#7
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Are you getting any errors in your PHP logs when you try and index via shell? Also, can you post a link to your PHP info? __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

01-11-2004, 08:35 PM	#8
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Try setting register_argc_argv to On in the php.ini file. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

01-12-2004, 06:56 PM	#9
Wayne McBryde Orange Mole Join Date: Oct 2003 Location: NC, USA Posts: 34	Charter, Changing register_argc_argv to On in the php.ini file did the trick. IT WORKS GREAT NOW!! THANKS __________________ Wayne Mcbryde http://LakeNormansWeb.com We search all of Lake Norman!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
command line indexing that actually works	carlaron	Troubleshooting	0	11-06-2006 09:48 PM
command line exec() pulls wrong file (php as cgi?)	kzant	Troubleshooting	2	02-01-2005 08:14 AM
Command line vs. admin indexing	wx3	Troubleshooting	8	09-08-2004 01:31 AM
Indexing by command line...	Canadian	How-to Forum	4	01-04-2004 07:44 PM
Indexing by command line interface	Skop	Troubleshooting	8	10-14-2003 03:23 AM