Geek Answers Handbook

Python - POST request for transferring data in several parts / data forms

I want to upload a file to a web server. From what I read, the best way to do this is to use the multipart / form-data encoding type in the HTTP POST request.

My research shows that there is no easy way to do this using the standard Python library. I am using Python 3.

(Note: see the requests package ( PyPI Link ) for easy implementation)

I am currently using this method:

import mimetypes, http.client boundary = 'wL36Yn8afVp8Ag7AmP8qZ0SA4n1v9T' # Randomly generated for fileName in fileList: # Add boundary and header dataList.append('--' + boundary) dataList.append('Content-Disposition: form-data; name={0}; filename={0}'.format(fileName)) fileType = mimetypes.guess_type(fileName)[0] or 'application/octet-stream' dataList.append('Content-Type: {}'.format(fileType)) dataList.append('') with open(fileName) as f: # Bad for large files dataList.append(f.read()) dataList.append('--'+boundary+'--') dataList.append('') contentType = 'multipart/form-data; boundary={}'.format(boundary) body = '\r\n'.join(dataList) headers = {'Content-type': contentType} conn = http.client.HTTPConnection('http://...') req = conn.request('POST', '/test/', body, headers) print(conn.getresponse().read())

This works to send text.

There are two problems: this is only text, and the entire text file must be stored in memory as a giant string.

How can I download any binary file? Is there a way to do this without reading the entire file in memory?

+4
source share
3 answers

I looked at this module

 class HTTPConnection: # ... def send(self, data): # line 820 """Send `data' to the server. ``data`` can be a string object, a bytes object, an array object, a file-like object that supports a .read() method, or an iterable object. """

data is exactly the body. You can pass the iterator as follows: (I have not tried)

 def body(): for fileName in fileList: # Add boundary and header yield('--' + boundary) + '\r\n' yield('Content-Disposition: form-data; name={0}; filename= {0}'.format(fileName)) + '\r\n' fileType = mimetypes.guess_type(fileName)[0] or 'application/octet-stream' yield('Content-Type: {}'.format(fileType)) + '\r\n' yield('\r\n') with open(fileName) as f: # Bad for large files yield f.read() yield('--'+boundary+'--') + '\r\n' yield('') + '\r\n'
+2
source

Take a look at the little Doug Hellmann urllib2 translated by me into python3.

I use it almost like this:

 import urllib.request import urllib.parse from lib.multipart_sender import MultiPartForm myfile = open('path/to/file', 'rb') form = MultiPartForm() form.add_field('token', apipost[mycgi['domain']]._token) form.add_field('domain', mycgi['domain']) form.add_file('file', 'logo.jpg', fileHandle=myfile) form.make_result() url = 'http://myurl' req1 = urllib.request.Request(url) req1.add_header('Content-type', form.get_content_type()) req1.add_header('Content-length', len(form.form_data)) req1.add_data(form.form_data) fp = urllib.request.urlopen(req1) print(fp.read()) # to view status
+2
source

You can use unirest to make a call. Code example

 import unirest # consume async post request def consumePOSTRequestSync(): params = {'test1':'param1','test2':'param2'} # we need to pass a dummy variable which is open method # actually unirest does not provide variable to shift between # application-x-www-form-urlencoded and # multipart/form-data params['dummy'] = open('dummy.txt', 'r') url = 'http://httpbin.org/post' headers = {"Accept": "application/json"} # call get service with headers and params response = unirest.post(url, headers = headers,params = params) print "code:"+ str(response.code) print "******************" print "headers:"+ str(response.headers) print "******************" print "body:"+ str(response.body) print "******************" print "raw_body:"+ str(response.raw_body) # post sync request multipart/form-data consumePOSTRequestSync()

Post a blog post for more details http://stackandqueue.com/?p=57

0
source

More articles:


All Articles