Not indexing pages, keywords, etc..

patrick@online- · 04-14-2006, 02:38 AM

Hi,

I've installed phpdig, and although it seems to go out crawling, all I ever
end up with are a database full of domains but no content, pages, or
anything else...

What gives?

If I use the search function, nothing ever shows there either, which is
because there's nothing to show, of course.

How do we get ahold of an older "working" version of this script? I have
tried the archive links, but they all lead to version 1.8.8.

Thanks,
Patrick

totobeloeil · 04-14-2006, 03:34 AM

Perhaps you have to change a value in admin/robot_functions.php :
replace $pu['port'] = '';
by $pu['port'] = 0;
as explained in this thread :

http://www.phpdig.net/forum/showthread.php?t=2350

I recently installed phpDig V1.8.8 and it's working

patrick@online- · 04-14-2006, 03:50 AM

Thanks for the reply,

I already tried that one, but still is not working properly.

Thanks anyway,
Patrick

totobeloeil · 04-14-2006, 04:11 AM

Perhaps this link could help you :

http://www.phpdig.net/forum/showthread.php?p=9414
If you modify php.ini, perhaps you have to restart the machine before it works.
Could be there are several php.ini files.
What is your configuration (windows / linux, etc...) ?
What are you trying to spider (localhost or external) ?
Did spider.php create anything in the sites table or not ?

patrick@online- · 04-14-2006, 02:12 PM

Hi,

Thanks again, but I'm on a Linux server with php.ini set per the install
instructions.

>What is your configuration (windows / linux, etc...) ?
Linux- Actually, using CentOS,and RedHat9 (tried on two separate servers)
but same basic deal.

>What are you trying to spider (localhost or external) ?
External Sites - Basically link pages and directories.

>Did spider.php create anything in the sites table or not ?
Just the domain (http://www.whatever.com/) but that's it.

As I look through the database structure, I really do not see any tables
or rows that could possibly hold web page content anyway... I'm wondering
if it's even been added to this version or maybe I have a bad database
set up?.?.

It also looks as if it's a pretty common problem as I look through the forums,
and no one really seems to have a solid answer to this issue...

I think I'll come back in a few months and see how the next release
comes along.

Thanks for the help,
Patrick

totobeloeil · 04-15-2006, 03:10 AM

Hi,
It seems that http://www.whatever.com/
has moved to http://www.funny.com/
With phpDig V1.8.8, I can't spider the first site, but the second yes.

04-14-2006, 02:38 AM	#1
patrick@online- Green Mole Join Date: Apr 2006 Posts: 3	Not indexing pages, keywords, etc.. Hi, I've installed phpdig, and although it seems to go out crawling, all I ever end up with are a database full of domains but no content, pages, or anything else... What gives? If I use the search function, nothing ever shows there either, which is because there's nothing to show, of course. How do we get ahold of an older "working" version of this script? I have tried the archive links, but they all lead to version 1.8.8. Thanks, Patrick

04-14-2006, 03:34 AM	#2
totobeloeil Green Mole Join Date: Apr 2006 Location: France Posts: 12	No spidering and robot_functions.php Perhaps you have to change a value in admin/robot_functions.php : replace $pu['port'] = ''; by $pu['port'] = 0; as explained in this thread : http://www.phpdig.net/forum/showthread.php?t=2350 I recently installed phpDig V1.8.8 and it's working

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
keywords missing after indexing	123av	Troubleshooting	2	10-21-2004 09:28 AM
excluding keywords from indexing	Fking	How-to Forum	1	10-05-2004 06:43 PM
Indexing finds all pages, but doesn't index all keywords	arakune	Troubleshooting	2	08-25-2004 06:18 PM
Reduce duplicates in keywords table through more intelligent indexing	jerrywin5	Mod Requests	1	04-20-2004 09:06 AM
Junk in keywords table - Indexing PDF	Bege	External Binaries	2	04-09-2004 08:15 AM

04-14-2006, 03:50 AM	#3
patrick@online- Green Mole Join Date: Apr 2006 Posts: 3	Thanks for the reply, I already tried that one, but still is not working properly. Thanks anyway, Patrick

04-14-2006, 04:11 AM	#4
totobeloeil Green Mole Join Date: Apr 2006 Location: France Posts: 12	Perhaps this link could help you : http://www.phpdig.net/forum/showthread.php?p=9414 If you modify php.ini, perhaps you have to restart the machine before it works. Could be there are several php.ini files. What is your configuration (windows / linux, etc...) ? What are you trying to spider (localhost or external) ? Did spider.php create anything in the sites table or not ?

04-14-2006, 02:12 PM	#5
patrick@online- Green Mole Join Date: Apr 2006 Posts: 3	Hi, Thanks again, but I'm on a Linux server with php.ini set per the install instructions. >What is your configuration (windows / linux, etc...) ? Linux- Actually, using CentOS,and RedHat9 (tried on two separate servers) but same basic deal. >What are you trying to spider (localhost or external) ? External Sites - Basically link pages and directories. >Did spider.php create anything in the sites table or not ? Just the domain (http://www.whatever.com/) but that's it. As I look through the database structure, I really do not see any tables or rows that could possibly hold web page content anyway... I'm wondering if it's even been added to this version or maybe I have a bad database set up?.?. It also looks as if it's a pretty common problem as I look through the forums, and no one really seems to have a solid answer to this issue... I think I'll come back in a few months and see how the next release comes along. Thanks for the help, Patrick

04-15-2006, 03:10 AM	#6
totobeloeil Green Mole Join Date: Apr 2006 Location: France Posts: 12	Hi, It seems that http://www.whatever.com/ has moved to http://www.funny.com/ With phpDig V1.8.8, I can't spider the first site, but the second yes.