can't index pdf using pdftotext - Page 2

rom · 08-12-2004, 01:26 PM

Hi Charter,

I read through the memory thread. Looked up my memory_limit, which is 10 M. Tried also this code for memory_get_usage from the php.net site:

<?php
// This is only an example, the numbers below will
// differ depending on your system
echo memory_get_usage() . "\n"; // 36640
$a = str_repeat("Hello", 4242);
echo memory_get_usage() . "\n"; // 57960
unset($a);
echo memory_get_usage() . "\n"; // 36744
?>

My server returned this:
16704 38000 16784

I know now, based on when the spidering ends, that it is getting hung up on one 4.6 M pdf.

From the memory thread, I wasn't sure what else to do, because at the end of the thread Tomas says nothing worked. Is there something that can be done to skip over that one file?

Thanks again.

Rom

Charter · 08-15-2004, 04:15 PM

Hi. Did you try something like in this post?

rom · 08-25-2004, 05:05 PM

hi charter,

tried your suggestion above. the indexing just stops part way through. seems to be when it encounters a 4.6 M file. it doesn't want to skip over it.

thanks,

rom

Charter · 08-25-2004, 08:10 PM

Hi. Did you try this too?

rom · 08-26-2004, 10:02 AM

Hi Charter,

Tried that also, but again it stops part way through indexing, when it reaches the 4.6 M file.

Thanks,

Rom

Charter · 08-26-2004, 11:21 AM

Assuming you are using 1.8.3, try moving this code:

PHP Code:


			
if (memory_get_usage() + 1000000 > 3000000) {

    return array('tempfile'=>0,'tempfilesize'=>0);

}

to be right after the following in the robot_functions.php file:

PHP Code:


			
// $file_content = @file($uri); /////////////////////////////////////////////////

rom · 08-26-2004, 02:55 PM

i'm using 1.8.0. should i upgrade first?

rom · 08-27-2004, 05:11 PM

I tried moving the lines as directed. Still stops indexing part way through at the same spot.

08-15-2004, 04:15 PM	#17
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Did you try something like in this post? __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

08-25-2004, 08:10 PM	#19
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Did you try this too? __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

08-26-2004, 11:21 AM	#21
Charter Head Mole Join Date: May 2003 Posts: 2,539	Assuming you are using 1.8.3, try moving this code: PHP Code: `if (memory_get_usage() + 1000000 > 3000000) { return array('tempfile'=>0,'tempfilesize'=>0); }` to be right after the following in the robot_functions.php file: PHP Code: `// $file_content = @file($uri); /////////////////////////////////////////////////` __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
PDF indexing Probelm (pdftotext)	ripchen	External Binaries	9	10-20-2005 12:14 PM
Can phpdig index Japanese PDF file???	mynamesucks	External Binaries	3	02-22-2005 10:59 PM
can phpdig index PDF server-side served from php?	Sybolt	How-to Forum	1	02-18-2005 01:16 PM
not indexing with pdftotext	davideyre	External Binaries	2	03-30-2004 01:55 PM
How to index a directory with pdf files	simonced	How-to Forum	3	02-13-2004 11:41 AM

08-12-2004, 01:26 PM	#16
rom Green Mole Join Date: Jan 2004 Posts: 25	Hi Charter, I read through the memory thread. Looked up my memory_limit, which is 10 M. Tried also this code for memory_get_usage from the php.net site: <?php // This is only an example, the numbers below will // differ depending on your system echo memory_get_usage() . "\n"; // 36640 $a = str_repeat("Hello", 4242); echo memory_get_usage() . "\n"; // 57960 unset($a); echo memory_get_usage() . "\n"; // 36744 ?> My server returned this: 16704 38000 16784 I know now, based on when the spidering ends, that it is getting hung up on one 4.6 M pdf. From the memory thread, I wasn't sure what else to do, because at the end of the thread Tomas says nothing worked. Is there something that can be done to skip over that one file? Thanks again. Rom

08-25-2004, 05:05 PM	#18
rom Green Mole Join Date: Jan 2004 Posts: 25	hi charter, tried your suggestion above. the indexing just stops part way through. seems to be when it encounters a 4.6 M file. it doesn't want to skip over it. thanks, rom

08-26-2004, 10:02 AM	#20
rom Green Mole Join Date: Jan 2004 Posts: 25	Hi Charter, Tried that also, but again it stops part way through indexing, when it reaches the 4.6 M file. Thanks, Rom

08-26-2004, 02:55 PM	#22
rom Green Mole Join Date: Jan 2004 Posts: 25	i'm using 1.8.0. should i upgrade first?

08-27-2004, 05:11 PM	#23
rom Green Mole Join Date: Jan 2004 Posts: 25	I tried moving the lines as directed. Still stops indexing part way through at the same spot.