Upgrade and Update

histrenact · 05-29-2007, 04:11 AM

Hi all,

Been looking at this site for ages but only really needed to post now. Have been using phpdig almost since the beginning but have been running 1.6.2 for about two years.

A bit about the site. I am one of the reference sites for phpdig and currently have approximately 500 sites indexed comprising just over twelve million index entries and just short of one and a half million keywords. The database is approximately 300Mb running on Linux Slackware 11, Apache 2.2.3 and Mysql 5.1.11 and php 5.2.0 all is fine with the current installation.

I have a couple of questions

1, Due to the fact that this is very much a production environment, the main website that I run which uses the search engine gets about a million hits a year or so, I tend to be very careful about updates. So I have a test envorinment which I am currently using to create the procedure to update the production site.

Running al of the update_to_xxx.sql from 1.6.2 upwards it appears that 1.8.6 and the most recent update_db.sql both have ALTER TABLE excludes DROP INDEX ex_id statements. Is this in error? After running 1.8.6 the index should have been removed?

2, I have seen reference here to the ability to run multiple spiders. I appreciate that this can be done with several lists of urls and several PCs so that the text areas do not clash but I have also seen reference to wrapper but the link to the download sesms to have gone. Can anyone help with the link?

3, MySQL NDB..... I am trying to get the database to an NDB cluster. Does anyone have any experience of this? Getting NDB working is no problem but with 12,000,000 three field records in the engine table, for example, is this the best way forwards? I get the impression from the documentation that illions of very small records may not be the best for using NDB, especially as the reason is to increase the search speed. Any suggestions for further "research" on my part would be very useful.

Finally, I do have all my own kit here, servers, software, complete control &c so if there is anything that *I* can do to help please feel free to ask.

David D.
The Historical Reenactment Web Site
http://www.histrenact.co.uk/

histrenact · 05-29-2007, 04:36 AM

Hmmm being a bit of a muppet on this one!

Having manually gone through all of the various database upgrades it appears that update_db_to_1_8_6.sql and update_db.sql are identical so thats sorted. However the documentation clearly states that any updates need to be done but that update_db.sql does the required updates from the last release so I workd on the assumption that this made changes from 1.8.6 to 1.8.8.

Hey Ho.

Take care

David D.
The Historical Reenactment Web Site
http://www.histrenact.co.uk

histrenact · 06-01-2007, 03:53 AM

Oh well, people looking at the thread but not answering......

Anyway practice upgrade went without a hitch which was very pleasing. However database update is *still* running after two weeks so will have to wait for that to finish before moving on.

Does *anyone* have any information about how to get multiple spiders working? It seems that the wrapper solution is the way to go but it also seems to have dissapeared off the boards....

Take care

David D.

05-29-2007, 04:11 AM	#1
histrenact Green Mole Join Date: May 2007 Posts: 3	Upgrade and Update Hi all, Been looking at this site for ages but only really needed to post now. Have been using phpdig almost since the beginning but have been running 1.6.2 for about two years. A bit about the site. I am one of the reference sites for phpdig and currently have approximately 500 sites indexed comprising just over twelve million index entries and just short of one and a half million keywords. The database is approximately 300Mb running on Linux Slackware 11, Apache 2.2.3 and Mysql 5.1.11 and php 5.2.0 all is fine with the current installation. I have a couple of questions 1, Due to the fact that this is very much a production environment, the main website that I run which uses the search engine gets about a million hits a year or so, I tend to be very careful about updates. So I have a test envorinment which I am currently using to create the procedure to update the production site. Running al of the update_to_xxx.sql from 1.6.2 upwards it appears that 1.8.6 and the most recent update_db.sql both have ALTER TABLE excludes DROP INDEX ex_id statements. Is this in error? After running 1.8.6 the index should have been removed? 2, I have seen reference here to the ability to run multiple spiders. I appreciate that this can be done with several lists of urls and several PCs so that the text areas do not clash but I have also seen reference to wrapper but the link to the download sesms to have gone. Can anyone help with the link? 3, MySQL NDB..... I am trying to get the database to an NDB cluster. Does anyone have any experience of this? Getting NDB working is no problem but with 12,000,000 three field records in the engine table, for example, is this the best way forwards? I get the impression from the documentation that illions of very small records may not be the best for using NDB, especially as the reason is to increase the search speed. Any suggestions for further "research" on my part would be very useful. Finally, I do have all my own kit here, servers, software, complete control &c so if there is anything that I can do to help please feel free to ask. David D. The Historical Reenactment Web Site http://www.histrenact.co.uk/

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Upgrade probs	Dave A	Script Installation	1	01-20-2006 04:44 AM
upgrade a 1.8.3 database to 1.8.6	renehaentjens	How-to Forum	6	01-06-2005 05:22 PM
New Upgrade	CrazyCanuck	Script Installation	9	07-26-2004 05:23 PM
Upgrade	xiodaos	Script Installation	1	07-19-2004 01:01 PM
vB Upgrade	Charter	Feedback & News	1	09-06-2003 12:20 PM

05-29-2007, 04:36 AM	#2
histrenact Green Mole Join Date: May 2007 Posts: 3	Hmmm being a bit of a muppet on this one! Having manually gone through all of the various database upgrades it appears that update_db_to_1_8_6.sql and update_db.sql are identical so thats sorted. However the documentation clearly states that any updates need to be done but that update_db.sql does the required updates from the last release so I workd on the assumption that this made changes from 1.8.6 to 1.8.8. Hey Ho. Take care David D. The Historical Reenactment Web Site http://www.histrenact.co.uk

06-01-2007, 03:53 AM	#3
histrenact Green Mole Join Date: May 2007 Posts: 3	Oh well, people looking at the thread but not answering...... Anyway practice upgrade went without a hitch which was very pleasing. However database update is still running after two weeks so will have to wait for that to finish before moving on. Does anyone have any information about how to get multiple spiders working? It seems that the wrapper solution is the way to go but it also seems to have dissapeared off the boards.... Take care David D.