PhpDig does not work (installs OK)

rafarspd · 11-14-2003, 07:33 AM

Here are the plain facts.

PhpDig Version 1.6.3
PHP Version 4.3.1
MySql Version 3.23.51
Apache 1.3.9 (Unix)
Server Protocol HTTP/1.1
CGI Version 1.1
Machine Type i386 Redhat-Linux-gnu
Hosting on Force9 (Plusnet Co UK)
PHP_MyAdmin Version 2.5.1
PHP_MyAdmin works OK
PHP_Nuke Version 6.9
PHP_Nuke works OK

Directories admin/temp, includes & text_content permissions set to 777
Database holds the following tables: engine, excludes, keywords,logs, sites, spider & tempspider.

Enter any url- e.g. http://www.rafars.org - or - http://www.rafars.org/filename - or - http://www.rafars.org/directory/filename where filename is *.html *.shtml *.txt
Select any level from 1 to 5 (not tried above this level).

After a wait the following is the only message received:-

Spidering in progress...

Site : http://www.rafars.org/
Exclude paths :
- @NONE@
No link in temporary table

links found : 0

...Was recently indexed
Optimizing tables...
Indexing Complete !

Database status shows:-
Hosts : 1 Entries
Pages : 0 Entries
Index : 0 Entries
Keywords : 0 Entries
Temporary Table : 0 Entries

The listed URL shows: http://www.rafars.org/ irrespective of the entry point.

There is one entry in the MySql database in 'sites'and that is the site URL 'http://www.rafars.org'.
There are no temporary files or changed files anywhere in the PHP_Dig directory structure.

Here are my emotive feelings.

I searched long for a site search engine to take the place of the CGI one I was using.
I found PhpDig, liked the look, liked the highlighting, liked the fact that I could decide which directory/files I could index.
My hopes were dashed as I feel that this program has a basic flaw insomuch as I think it would be OK on a local site but fails when on a site hosted with an ISP.
I also feel the documentation could be improved to help people with worn out grey matter (like me - 65 and white hair - hopefully not past it!) .
I live in hope that someone will prove me wrong, advise me of my errors so I can use PhpDig.

Bob Finch
Webmaster www.rafars.org

Charter · 11-14-2003, 05:18 PM

Hi. Thanks for the comments.

PhpDig can be used with sites hosted with an ISP, but there seems to be some problem with PhpDig when used in CGI mode, besides possible timeout issues.

I currently don't have access to PHP in CGI mode. If someone out there, who is having problems like those posted above, can offer access, then I could try to locate the problem.

rafarspd · 11-15-2003, 12:10 AM

Hi

I could not answer you question abour CGI mode, beyond my knowledge.

PhpDig is located outside the cgi-bin directory, according to the ISP you cannot run CGI programs outside this directory.

I have attached the results of phpinfo.php, it may help.

Regards

Charter · 11-15-2003, 06:44 AM

Hi. Does PhpDig work when you install it in the cgi-bin directory?

Charter · 11-15-2003, 02:31 PM

Hi. I purchased a hosting account that runs PHP in CGI mode, and I also set an open_basedir restriction. I was able to crawl several sites using this account without incident, including your rafars.org site. Please install PhpDig in the CGI directory, specifically in the open_basedir directory if set, and let me know how it goes.

renehaentjens · 11-17-2003, 07:28 AM

Maybe not related, but...

I had similar problems, and moreover also:
Fatal error: Call to undefined function: is_executable() in ...\admin\robot_functions.php on line 633.

When I changed USE_IS_EXECUTABLE_COMMAND to '0' in the config file, my site started indexing properly.

rafarspd · 11-19-2003, 05:19 AM

Hi

Charter
PhpDig does not work inside the cgi-bin directory.
Not even 'phpinfo.php', it returns error 404 and when permissions changed to 755 it returns error 500.

'open_basedir' is set to 'no value' at the ISP.

renehaentjens
I do not have any files that include 'USE_IS_EXECUTEABLE_COMMAND' (I searched for USE_IS).

[Just cleared the files and installed Version 1.6.4 so able to try the USE_IS command, just the same].

I have set every php file to 755 (except those that were set to 777) - still not working.

What is the correct permissions for the directories and files! - I may have a misunderstanding with these.

Is there a command to echo errors to the browser ( I know what it is in cgi). This would help in pinpointing where the failure is.

p.s. The Avatar shows me smoking a pipe, it has a flue brush poking out of the top of the pipe.
For those who do not know - pipe smokers are renown for their creation of burnt offerings and regular dekoking of the bowl.

Charter · 11-19-2003, 07:42 AM

Hi. The PHP scripts should work at 644. The following directories should be 777: /text_content, /includes, and /admin/temp. What do you see if you put error_reporting(E_ALL); at the top of the spider.php file and then crawl a site that hasn't been crawled before?

rafarspd · 11-20-2003, 03:56 AM

Hello again.
The permissions are set as specified above.
error_reporting(E_ALL); does not report any errors.

PROGRESS REPORT.
PhpDig does work! It does not crawl my site.
I can crawl other sites OK, so I beleive that the nill report shown in my original post is trying to tell me that it cannot resolve the URL of my site.

I have posted a support request with my ISP and will promulgate the reply.

Thanks for the time and effort from all.

rafarspd · 11-27-2003, 01:30 AM

I thought you would like a progress report.

My ISP uses two servers, one for normal html (called the homepages server) and one dedicated to CGI, PHP, MySql etc.

PhpDig resides on the CGI server.
It can crawl any site on the homepages server but fails on any site on the CGI server.
This has been proven by installing PhpDig under my own user name account in addition to the www.rafars.org account.

Here is the reply from my ISP:-

cut -----
This is normal, the CGI platform is a loadbalanced cluster of servers, it is therefore not possible to connect back to "itself" on the loadbalanced address.

We are investigating other possibilities to enable this for the use of htdig etc although there is no ETA for this.
---- cut

So at least an answer from the ISP.

I say again:-
NOT A PHPDIG PROBLEM but an ISP CONFIGURATION PROBLEM.

Charter, can you please change the 'thumbs down' to a 'thumbs up' on the heading.

Charter · 11-27-2003, 07:39 AM

Quote:

Originally posted by rafarspd
Charter, can you please change the 'thumbs down' to a 'thumbs up' on the heading.

Changed, and thanks for the updates.

hess_matt · 12-21-2003, 10:52 AM

I was having the same problem that you discribed. I spend an hour looking through the PHPDIG code and found no appearant problems. I made a change to the config.php file; changing the PHPDIG_DEFAULT_INDEX to true. This still did not resolve the problem. I then thought there may exist a connection problem. I edited the HOSTS file with the IP and host name that I was trying to index. (BTW: I am on a Linux box, Apache, MySQL 3.23) Everything started to work.

I hope this helps you with your problem.

dosteen · 01-06-2004, 04:20 PM

This worked for me too.

11-14-2003, 07:33 AM	#1
rafarspd Orange Mole Join Date: Nov 2003 Location: High Wycombe, Bucks. UK Posts: 35	PhpDig does not work (installs OK) Here are the plain facts. PhpDig Version 1.6.3 PHP Version 4.3.1 MySql Version 3.23.51 Apache 1.3.9 (Unix) Server Protocol HTTP/1.1 CGI Version 1.1 Machine Type i386 Redhat-Linux-gnu Hosting on Force9 (Plusnet Co UK) PHP_MyAdmin Version 2.5.1 PHP_MyAdmin works OK PHP_Nuke Version 6.9 PHP_Nuke works OK Directories admin/temp, includes & text_content permissions set to 777 Database holds the following tables: engine, excludes, keywords,logs, sites, spider & tempspider. Enter any url- e.g. http://www.rafars.org - or - http://www.rafars.org/filename - or - http://www.rafars.org/directory/filename where filename is .html .shtml *.txt Select any level from 1 to 5 (not tried above this level). After a wait the following is the only message received:- Spidering in progress... Site : http://www.rafars.org/ Exclude paths : - @NONE@ No link in temporary table links found : 0 ...Was recently indexed Optimizing tables... Indexing Complete ! Database status shows:- Hosts : 1 Entries Pages : 0 Entries Index : 0 Entries Keywords : 0 Entries Temporary Table : 0 Entries The listed URL shows: http://www.rafars.org/ irrespective of the entry point. There is one entry in the MySql database in 'sites'and that is the site URL 'http://www.rafars.org'. There are no temporary files or changed files anywhere in the PHP_Dig directory structure. Here are my emotive feelings. I searched long for a site search engine to take the place of the CGI one I was using. I found PhpDig, liked the look, liked the highlighting, liked the fact that I could decide which directory/files I could index. My hopes were dashed as I feel that this program has a basic flaw insomuch as I think it would be OK on a local site but fails when on a site hosted with an ISP. I also feel the documentation could be improved to help people with worn out grey matter (like me - 65 and white hair - hopefully not past it!) . I live in hope that someone will prove me wrong, advise me of my errors so I can use PhpDig. Bob Finch Webmaster www.rafars.org __________________ Regards Bob Finch

11-14-2003, 05:18 PM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Thanks for the comments. PhpDig can be used with sites hosted with an ISP, but there seems to be some problem with PhpDig when used in CGI mode, besides possible timeout issues. I currently don't have access to PHP in CGI mode. If someone out there, who is having problems like those posted above, can offer access, then I could try to locate the problem. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

11-15-2003, 06:44 AM	#4
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Does PhpDig work when you install it in the cgi-bin directory? __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

11-15-2003, 02:31 PM	#5
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. I purchased a hosting account that runs PHP in CGI mode, and I also set an open_basedir restriction. I was able to crawl several sites using this account without incident, including your rafars.org site. Please install PhpDig in the CGI directory, specifically in the open_basedir directory if set, and let me know how it goes. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

11-17-2003, 07:28 AM	#6
renehaentjens Orange Mole Join Date: Nov 2003 Posts: 69	Maybe not related, but... I had similar problems, and moreover also: Fatal error: Call to undefined function: is_executable() in ...\admin\robot_functions.php on line 633. When I changed USE_IS_EXECUTABLE_COMMAND to '0' in the config file, my site started indexing properly. __________________ René Haentjens, Ghent University

11-19-2003, 05:19 AM	#7
rafarspd Orange Mole Join Date: Nov 2003 Location: High Wycombe, Bucks. UK Posts: 35	Hi Charter PhpDig does not work inside the cgi-bin directory. Not even 'phpinfo.php', it returns error 404 and when permissions changed to 755 it returns error 500. 'open_basedir' is set to 'no value' at the ISP. renehaentjens I do not have any files that include 'USE_IS_EXECUTEABLE_COMMAND' (I searched for USE_IS). [Just cleared the files and installed Version 1.6.4 so able to try the USE_IS command, just the same]. I have set every php file to 755 (except those that were set to 777) - still not working. What is the correct permissions for the directories and files! - I may have a misunderstanding with these. Is there a command to echo errors to the browser ( I know what it is in cgi). This would help in pinpointing where the failure is. p.s. The Avatar shows me smoking a pipe, it has a flue brush poking out of the top of the pipe. For those who do not know - pipe smokers are renown for their creation of burnt offerings and regular dekoking of the bowl. __________________ Regards Bob Finch Last edited by rafarspd; 11-19-2003 at 07:14 AM.

11-19-2003, 07:42 AM	#8
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. The PHP scripts should work at 644. The following directories should be 777: /text_content, /includes, and /admin/temp. What do you see if you put error_reporting(E_ALL); at the top of the spider.php file and then crawl a site that hasn't been crawled before? __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

11-20-2003, 03:56 AM	#9
rafarspd Orange Mole Join Date: Nov 2003 Location: High Wycombe, Bucks. UK Posts: 35	Hello again. The permissions are set as specified above. error_reporting(E_ALL); does not report any errors. PROGRESS REPORT. PhpDig does work! It does not crawl my site. I can crawl other sites OK, so I beleive that the nill report shown in my original post is trying to tell me that it cannot resolve the URL of my site. I have posted a support request with my ISP and will promulgate the reply. Thanks for the time and effort from all. __________________ Regards Bob Finch

11-27-2003, 01:30 AM	#10
rafarspd Orange Mole Join Date: Nov 2003 Location: High Wycombe, Bucks. UK Posts: 35	I thought you would like a progress report. My ISP uses two servers, one for normal html (called the homepages server) and one dedicated to CGI, PHP, MySql etc. PhpDig resides on the CGI server. It can crawl any site on the homepages server but fails on any site on the CGI server. This has been proven by installing PhpDig under my own user name account in addition to the www.rafars.org account. Here is the reply from my ISP:- cut ----- This is normal, the CGI platform is a loadbalanced cluster of servers, it is therefore not possible to connect back to "itself" on the loadbalanced address. We are investigating other possibilities to enable this for the use of htdig etc although there is no ETA for this. ---- cut So at least an answer from the ISP. I say again:- NOT A PHPDIG PROBLEM but an ISP CONFIGURATION PROBLEM. Charter, can you please change the 'thumbs down' to a 'thumbs up' on the heading. __________________ Regards Bob Finch

12-21-2003, 10:52 AM	#12
hess_matt Green Mole Join Date: Dec 2003 Posts: 1	No link in temporary table fix I was having the same problem that you discribed. I spend an hour looking through the PHPDIG code and found no appearant problems. I made a change to the config.php file; changing the PHPDIG_DEFAULT_INDEX to true. This still did not resolve the problem. I then thought there may exist a connection problem. I edited the HOSTS file with the IP and host name that I was trying to index. (BTW: I am on a Linux box, Apache, MySQL 3.23) Everything started to work. I hope this helps you with your problem.

01-06-2004, 04:20 PM	#13
dosteen Green Mole Join Date: Jan 2004 Posts: 1	This worked for me too.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Anyone considering making PhpDig work with SQL rather than MySQL?	misterbearcom	Mod Requests	0	08-10-2005 03:25 PM
PhpDig indexing won't work	sigfy	Troubleshooting	11	01-07-2005 06:47 AM
Cronjob for spidering doen't work anymore with PhpDig 1.8.6	gaam	Troubleshooting	0	12-22-2004 12:28 AM
Install phpdig in a file named phpdig doesn't work	Sansnom	Script Installation	1	05-09-2004 03:13 PM
pdftotext with phpdig does not work	tomas	External Binaries	14	02-25-2004 01:45 PM