PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > How-to Forum

Reply
 
Thread Tools
Old 12-13-2003, 05:06 PM   #1
Not Logged In
Green Mole
 
Not Logged In's Avatar
 
Join Date: Nov 2003
Posts: 5
Limit number of spidered pages

I would like to put a limit on the number of pages that are indexed, save on some space. LIMIT 15?

thanks,
Robert
Not Logged In is offline   Reply With Quote
Old 12-14-2003, 02:27 PM   #2
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. The below has not been tested. When I get a chance I can test it, but for now it gives you something to try.

In spider.php find the following:
PHP Code:
// Spidering ...
while($level <= $limit) { 
and replace with the following:
PHP Code:
// Spidering ...
$my_spider_limit 0;
while((
$level <= $limit) && ($my_spider_limit 15)) {
$my_spider_limit++; 
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 12-15-2003, 04:51 PM   #3
Not Logged In
Green Mole
 
Not Logged In's Avatar
 
Join Date: Nov 2003
Posts: 5
Sorry to say that didn't work


I've tried to limit the number of times the tempspider is written but can't figure it out.


Any ideas?

Thanks for the help!
Not Logged In is offline   Reply With Quote
Old 12-16-2003, 10:18 AM   #4
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. Please undo the above changes and then see if this thread helps.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 12-16-2003, 02:10 PM   #5
fr :: anonymus
Green Mole
 
fr :: anonymus's Avatar
 
Join Date: Dec 2003
Location: Lyon, France
Posts: 17
Charter : A bug in you code =>

// Spidering ...
$my_spider_limit = 0;
while(($level <= $limit) && ($my_spider_limit < 15)) {
$my_spider_limit++;

You must change && by OR :

// Spidering ...
$my_spider_limit = 0;
while(($level <= $limit) OR ($my_spider_limit < 15)) {
$my_spider_limit++;

Spider must stop when
$level <= $limit
OR when
$my_spider_limit <15

Isn't it ?
fr :: anonymus is offline   Reply With Quote
Old 12-16-2003, 03:03 PM   #6
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539

Quote:
Originally posted by fr :: anonymus
Charter : A bug in you code =>

// Spidering ...
$my_spider_limit = 0;
while(($level <= $limit) && ($my_spider_limit < 15)) {
$my_spider_limit++;

You must change && by OR :

// Spidering ...
$my_spider_limit = 0;
while(($level <= $limit) OR ($my_spider_limit < 15)) {
$my_spider_limit++;

Spider must stop when
$level <= $limit
OR when
$my_spider_limit <15

Isn't it ?

Hi. Thanks, but in that code I wanted the 'and' in place.

It was meant as when $level <= $limit and when $my_spider_limit < 15 do the loop, but it was the wrong piece of code to edit because all it does is basically limit the seach depth to fourteen without affecting the number of links found in a page. Silly me.

Please see this thread instead.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Number of sites locked/being spidered CentaurAtlas Mod Submissions 0 12-09-2006 02:38 AM
Show number of indexed pages on homepage Freddie Mod Submissions 5 01-18-2005 04:36 PM
Number of pages indexed claudiomet How-to Forum 0 08-30-2004 02:26 PM
Set time limit on spider.php or number of pages paullind Troubleshooting 1 05-01-2004 07:25 AM
pages number limited indexing JÿGius³ Mod Requests 4 01-13-2004 11:17 AM


All times are GMT -8. The time now is 07:15 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.