Passing the "+" character in a POST request in Python

Question

I am trying to do some automation in a Python script, and I ran into a problem. I am trying to do a POST on a server.

url = 'http://www.example.com' params = {'arg0': 'value', 'arg1': '+value'} f = urllib.urlopen(url, urllib.urlencode(params)) print f.read() 

I did a proxy capture of the equivalent browser operation where the second arg, arg1 is passed as +value , however when I do this with Python, + changes to %2B , i.e.

 Line-based text data: application/x-www-form-urlencoded arg0=value&arg1=%2Bvalue 

when it should be:

 Line-based text data: application/x-www-form-urlencoded arg0=value&arg1=+value 

I also used the Requests module and it seems to have done the same.

 url = 'http://www.example.com' params = {'arg0': 'value', 'arg1': '+value'} f = requests.post(url, params) 

Google is not your friend when you have a "+" problem, as it seems to be a trick for everything else.

Answer 1

The + character is the correct space encoding when quoting GET or POST data. Thus, the letter symbol + must also be escaped so that it is not decoded by the space at the other end. See RFC 2396, section 2.2 , section 3.4, and the HTML specification, application/x-www-form-urlencoded section :

Control names and values ​​are escaped. Space characters are replaced with `+ ', and then reserved characters are escaped as described in [RFC1738] , section 2.2.

If you send data to an application that does not decode the + character for a space, but instead processes data such as plus signs, you need to encode your parameters yourself using urllib.quote , indicating that the + character should not be encoded:

 import urllib def urlencode_withoutplus(query): if hasattr(query, 'items'): query = query.items() l = [] for k, v in query: k = urllib.quote(str(k), safe=' /+') v = urllib.quote(str(v), safe=' /+') l.append(k + '=' + v) return '&'.join(l) 

Demo:

 >>> urlencode_withoutplus({'arg0': 'value', 'arg1': '+value'}) 'arg0=value&arg1=+value' 

When using requests you can simply pass the result of the specified function as the value of data , but in this case you need to manually set the content type:

 requests.post(url, urlencode_withoutplus(query), headers={'Content-Type': 'application/x-www-form-urlencoded'}) 

Answer 2

 urllib2.quote(' ') # '%20' urllib2.unquote('%20') # ' ' 

So, why not just specify the parameter number:

 f = urllib.urlopen(url, urllib.unquote(urllib.urlencode(params)))