http - Why am I getting this error in python ? (httplib)

Question

if theurl.startswith("http://"): theurl = theurl[7:]
    head = theurl[:theurl.find('/')]
    tail = theurl[theurl.find('/'):]
response_code = 0
import httplib
conn = httplib.HTTPConnection(head)
conn.request("HEAD",tail)
res = conn.getresponse()
response_code = int(res.status)

http://www.garageband.com/mp3cat/.UZCKbS6N4qk/01_Saraenglish.mp3
Traceback (most recent call last):
  File "check_data_404.py", line 51, in <module>
    run()
  File "check_data_404.py", line 35, in run
    res = conn.getresponse()
  File "/usr/lib/python2.6/httplib.py", line 950, in getresponse
    response.begin()
  File "/usr/lib/python2.6/httplib.py", line 390, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.6/httplib.py", line 354, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine

Does anyone know what "Bad Status Line" is?

Edit: I tried this for many servers, and many URL's and I still get this error?

Answer 1

From the documentation for httplib (Python 2) (called http.client in Python 3):

exception httplib.BadStatusLine: (exception http.client.BadStatusLine:)

A subclass of HTTPException.

Raised if a server responds with an HTTP status code that we don’t understand.

I ran the same code and did not receive an error:

>>> theurl = 'http://www.garageband.com/mp3cat/.UZCKbS6N4qk/01_Saraenglish.mp3'
>>> if theurl.startswith("http://"):
...     theurl = theurl[7:]
...     head = theurl[:theurl.find('/')]
...     tail = theurl[theurl.find('/'):]
... 
>>> head
'www.garageband.com'
>>> tail
'/mp3cat/.UZCKbS6N4qk/01_Saraenglish.mp3'
>>> response_code = 0
>>> import httplib
>>> conn = httplib.HTTPConnection(head)
>>> conn.request("HEAD", tail)
>>> res = conn.getresponse()
>>> res.status
302
>>> response_code = int(res.status)

I guess just double-check everything and try again?

Answer 2

I recently got this error, in a situation where the method that contained the http request ran successfully once, and then threw this exception (with the status code as an empty string) the second time the method was called (with a different URL). I had a debugging advantage because this is calling my own REST api, so I did some logging on the server side and discovered that the request was never being received. I ultimately figured out that my URL string had a trailing newline character. So make sure that your URLs are stripped of any leading or trailing special characters.

Answer 3

The Python Standard Library: httplib (Python 2) (called http.client in Python 3):

exception httplib.BadStatusLine
A subclass of HTTPException. Raised if a server responds with a HTTP status code that we don’t understand.

Answer 4

Are you using a proxy?

If so, perhaps the proxy server is rejecting HEAD requests.

Do you get the same problem if you issue a GET request? If GET works I'd suspect that there is a proxy in your way.

You can see what's going on in more detail by calling conn.set_debuglevel(1) prior to calling conn.request(...).

Answer 5

I just found when we get exception httplib.BadStatusLine , is when server goes down and doesn't send any response, so it means web server doesn't even send the http code [1]

[1] http://en.wikipedia.org/wiki/List_of_HTTP_status_codes

Answer 6

The problem I had was with multiple requests, but BadStatusLine was only occurring between requests with more then a 5 second interval with a Keep-Alive timeout=5. I'm still uncertain why BadStatusLine was being raised instead of NotConnected. It seems that the connection also defaults to 5 when the header is missing. The fix was conn.connect() before each request.

Answer 7

I've also encountered this problem. Accordig to GmailAPI (python), it happens when the server closes the connection before sending a valid respone. Indeed, this only happened to me when my program ran on large DB.

def _read_status(self):
    # Initialize with Simple-Response defaults
    line = self.fp.readline(_MAXLINE + 1)
    if len(line) > _MAXLINE:
        raise LineTooLong("header line")
    if self.debuglevel > 0:
        print "reply:", repr(line)
    if not line:
        # Presumably, the server closed the connection before
        # sending a valid response.
        raise BadStatusLine(line)

My solution was to move all the part that establishes the connection with gmail into a function. Then, call this function only before actually sending the email. Before that, the parts incharge of establishing the connection were just 'thrown' in some .py file, and therefore were called at the begining of the run.

Answer 8

I know that "you should just use X" answers are frowned upon, but I have to say that after trying to diagnose this same problem for a couple hours I tried Requests with the same set up and it worked perfectly. Easier to use and debug in my opinion as well.

Answer 9

We have no clues about what is in your theurl string, and I do not know whether your problem is solved or not (6 years have past and I hope you made it long before), so I just give you one possible reason I met and share it with someone who may find this later.

I have encountered a quite similar problem, in which my code runs quite well on some computers but raises BadStatusLine exceptions sometimes. It is quite annoying just like a ghost.

After careful checked all the possible stituation, I found a 'Content-Length' component was in my request http header. After removeing the component, my code runs well in all computers. Maybe the first part of your theurl contains something like mine, which contradicts the true data length.

Answer 10

I saw this error when trying to access a HTTPS/SSL URL using httplib.HTTPConnection

You should use httplib.HTTPSConnection to access SSL urls.