Requests, can't assign the requested address, from ports?

Question

requests.exceptions.ConnectionError: ('Connection aborted.', error(99, 'Cannot assign requested address')) 

I got this error when starting several processes using the python request library and calling the post function in the API, which returned very quickly (<10ms).

Reducing the number of running processes had a delay effect, but only a set of up to 1 process fixed the problem. This was not a solution, but indicated that the end resource was the culprit.

Answer 1

As I solved, the problem was to use the requests.Session class, where I would reuse the same connection / session for every call in this process.

A simplified example:

 import requests for line in file: requests.get('http://example.com/api?key={key}'.format(key=line['key'])) 

becomes

 import requests with requests.Session() as session: for line in file: session.get('http://example.com/api?key={key}'.format(key=line['key'])) 

These questions had some recommendations:

Repeated POST request causes error "socket.error: (99, 'Unable to assign the requested address')" Python urllib2: Unable to assign the requested address

Answer 2

I also came across a similar problem when executing several POST instructions using the python request library in Spark. To make matters worse, I used multiprocessing for each artist to send to the server. Thus, thousands of connections created in seconds took several seconds to change state from TIME_WAIT and free up ports for the next set of connections.

Of all the available solutions available on the Internet that talk about disabling keep-alive using with request.Session () and others, I found this answer to work. You may need to put the contents of the header on the separte line outside the command line.

 headers = { 'Connection': 'close' } with requests.Session() as session: response = session.post('https://xx.xxx.xxx.x/xxxxxx/x', headers=headers, files=files, verify=False) results = response.json() print results