Geek Answers Handbook

Python POST HTTPS Request

I want to make an email request to an HTTPS site that should respond with a CSV file. I have this Python code:

url = 'https://www.site.com/servlet/datadownload' values = { 'val1' : '123', 'val2' : 'abc', 'val3' : '1b3', } data = urllib.urlencode(values) req = urllib2.Request(url,data) response = urllib2.urlopen(req) myfile = open('file.csv', 'wb') shutil.copyfileobj(response.fp, myfile) myfile.close()

But we get the error:

 BadStatusLine: '' (in httplib.py)

I tried the mail request with the Chrome Extension: Advanced REST client (screenshot) and it works great.

What could be the problem and how can I solve it? (is this related to https?)

EDIT Reorganized Code:

 try: #conn = httplib.HTTPSConnection(host="www.site.com", port=443)

=> Gives BadStatusLine: '' error BadStatusLine: ''

  conn = httplib.HTTPConnection("www.site.com"); params = urllib.urlencode({'val1':'123','val2':'abc','val3':'1b3'}) conn.request("POST", "/nps/servlet/exportdatadownload", params) content = conn.getresponse() print content.reason, content.status print content.read() conn.close() except: import sys print sys.exc_info()[:2]

Output:

 Found 302 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <HTML><HEAD> <TITLE>302 Found</TITLE> </HEAD><BODY> <H1>Found</H1> The document has moved <A HREF="https://www.site.com/nps/servlet/exportdatadownload">here</A>.<P> <HR> <ADDRESS>Oracle-Application-Server-10g/10.1.3.5.0 Oracle-HTTP-Server Server at mp-www1.mrco.be Port 7778</ADDRESS> </BODY></HTML>

What am I doing wrong?

+8
python post urllib2 urllib
source share
4 answers

BadStatusLine: '' (in httplib.py) gives that there may be something else. This can happen when the server does not send a response at all and just closes the connection.

As you mentioned that you are using an SSL connection, it can be especially interesting to debug (if necessary, curl -v URL ). If you find that curl -2 URL (which forcibly uses SSLv2) seems to work, but curl -3 URL (SSLv3) does not, you can take a look at question # 13636 and possibly # 11220 in python bugtracker. Depending on your version of Python and possibly a misconfigured web server, this can cause problems: SSL defaults have been changed in version 2.2.

+2
source

Is there a reason you need to use urllib ? Requests is simpler, better in almost every way, and abstracts the part of the crack that urllib is hard to work urllib .

As an example, I would rework your example like:

 import requests resp = requests.post(url, data=values, allow_redirects=True)

At this point, the response from the server is available in resp.text , and you can do what you want. If the requests were unable to correctly configure the POST (for example, you need a special SSL certificate), it should give you a good error message that tells you why.

Even if you cannot do this in your production environment, do it in the local shell to find out what error messages you receive from requests , and use this to debug urllib .

+9
source 
  conn = httplib.HTTPSConnection(host='www.site.com', port=443, cert_file=_certfile) params = urllib.urlencode({'cmd': 'token', 'device_id_st': 'AAAA-BBBB-CCCC', 'token_id_st':'DDDD-EEEE_FFFF', 'product_id':'Unit Test', 'product_ver':"1.6.3"}) conn.request("POST", "servlet/datadownload", params) content = conn.getresponse().read() #print response.status, response.reason conn.close()
+1
source

The server may not like the missing headers, especially the user agent and content type. The Chrome image shows what is used for them. Perhaps try adding headers:

 import httplib, urllib host = 'www.site.com' url = '/servlet/datadownload' values = { 'val1' : '123', 'val2' : 'abc', 'val3' : '1b3', } headers = { 'User-Agent': 'python', 'Content-Type': 'application/x-www-form-urlencoded', } values = urllib.urlencode(values) conn = httplib.HTTPSConnection(host) conn.request("POST", url, values, headers) response = conn.getresponse() data = response.read() print 'Response: ', response.status, response.reason print 'Data:' print data

This is untested code, and you can experiment by adding other header values ​​to match your screenshot. Hope this helps.

0
source

More articles:


All Articles