getting '400 Bad Request' in some search results titles

jdell_nv · 04-07-2005, 02:28 PM

Hi,

I saw another thread about this problem relating to XHTML and some other stuff, but I don't have any of that.

If you have a look at my search site http://casat.unr.edu/search/
and enter search term of 'FRN' (you can also try 'PIC') you will see results with this in the title. However, I've reviewed the pages very carefully and can't see any bad links or other problems that might cause this.

The only thing I can think to suspect is that I have 2 similar URL's:

http://casat.unr.edu/frn/
http://casat.unr.edu/frn.html

and I have the setting PHPDIG_DEFAULT_INDEX set to TRUE.

Any ideas?

Best Regards,
John

jdell_nv · 04-07-2005, 02:32 PM

Also, I'm not seeing any errors in the spider.log

Charter · 04-08-2005, 07:50 AM

It looks like the server is not happy with certain requests, don't know why, but here's a quick and dirty way to get that out of the title.

In the phpdigCleanHtml function of robot_functions.php find:

Code:

$title = trim($regs[1]);

And replace with something like the following:

Code:

$title = trim(str_replace("400 Bad Request","",$regs[1]));

jdell_nv · 04-08-2005, 10:55 AM

Cool! Thanks for the quick/dirty fix.

Do you have any suggestions for how I might try to debug the real cause of the problem? I'm happy to dig in and I'm fairly proficient with PHP. Server is Linux/Apache 1.3.33/PHP 4.3.10

Regards,
John

Charter · 04-08-2005, 04:10 PM

Hmm, perhaps start with the phpdigGetUrl function, echo the requests, and try them by hand via shell.

04-07-2005, 02:28 PM	#1
jdell_nv Green Mole Join Date: Oct 2003 Posts: 8	getting '400 Bad Request' in some search results titles Hi, I saw another thread about this problem relating to XHTML and some other stuff, but I don't have any of that. If you have a look at my search site http://casat.unr.edu/search/ and enter search term of 'FRN' (you can also try 'PIC') you will see results with this in the title. However, I've reviewed the pages very carefully and can't see any bad links or other problems that might cause this. The only thing I can think to suspect is that I have 2 similar URL's: http://casat.unr.edu/frn/ http://casat.unr.edu/frn.html and I have the setting PHPDIG_DEFAULT_INDEX set to TRUE. Any ideas? Best Regards, John

04-08-2005, 07:50 AM	#3
Charter Head Mole Join Date: May 2003 Posts: 2,539	It looks like the server is not happy with certain requests, don't know why, but here's a quick and dirty way to get that out of the title. In the phpdigCleanHtml function of robot_functions.php find: Code: $title = trim($regs[1]); And replace with something like the following: Code: $title = trim(str_replace("400 Bad Request","",$regs[1])); __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

04-08-2005, 04:10 PM	#5
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hmm, perhaps start with the phpdigGetUrl function, echo the requests, and try them by hand via shell. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
TitlePageLink 400 Bad Request	flash7	Troubleshooting	5	11-03-2004 08:55 AM
how to show shorter titles in results?	orbitalz	Troubleshooting	1	05-01-2004 06:18 AM
help request for design and development of a search engine	tonguim	The Mole Hole	4	04-07-2004 05:20 AM
User Titles	Charter	The Mole Hole	0	01-30-2004 10:55 PM

04-07-2005, 02:32 PM	#2
jdell_nv Green Mole Join Date: Oct 2003 Posts: 8	Also, I'm not seeing any errors in the spider.log

04-08-2005, 10:55 AM	#4
jdell_nv Green Mole Join Date: Oct 2003 Posts: 8	Cool! Thanks for the quick/dirty fix. Do you have any suggestions for how I might try to debug the real cause of the problem? I'm happy to dig in and I'm fairly proficient with PHP. Server is Linux/Apache 1.3.33/PHP 4.3.10 Regards, John