PhpDig.net - View Single Post - why is a temptable so big? (phpdigtempspider)

Charter · 12-20-2003, 06:57 PM

Hi. Each keyword is associated with an id in the keywords table. These ids are associated with the text files in the engine table. With a large engine table, it seems like there are a lot of the same keywords showing up in different pages. The number of rows in the sites table is how many sites were crawled. The number of rows in the spider table is how many pages were crawled.

The tempspider may contain rows when the spidering process is stopped prematurely. These rows may be removed when the associated site is reindexed. The tempspider table may also contain rows even after a site has been indexed. The latter is something that needs further investigation. In the meantime, just make a backup of your database, and then afterwards empty the tempspider table if you want.

12-20-2003, 06:57 PM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Each keyword is associated with an id in the keywords table. These ids are associated with the text files in the engine table. With a large engine table, it seems like there are a lot of the same keywords showing up in different pages. The number of rows in the sites table is how many sites were crawled. The number of rows in the spider table is how many pages were crawled. The tempspider may contain rows when the spidering process is stopped prematurely. These rows may be removed when the associated site is reindexed. The tempspider table may also contain rows even after a site has been indexed. The latter is something that needs further investigation. In the meantime, just make a backup of your database, and then afterwards empty the tempspider table if you want. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.