Geek Answers Handbook

Repeated POST request causes error "socket.error: (99," Unable to assign the requested address ")

I have a web service deployed in my inbox. I want to check the result of this service with various inputs. Here is the code I'm using:

import sys import httplib import urllib apUrl = "someUrl:somePort" fileName = sys.argv[1] conn = httplib.HTTPConnection(apUrl) titlesFile = open(fileName, 'r') try: for title in titlesFile: title = title.strip() params = urllib.urlencode({'search': 'abcd', 'text': title}) conn.request("POST", "/somePath/", params) response = conn.getresponse() data = response.read().strip() print data+"\t"+title conn.close() finally: titlesFile.close()

This code gives an error after as many lines are printed (28233). Error message:

 Traceback (most recent call last): File "testService.py", line 19, in ? conn.request("POST", "/somePath/", params) File "/usr/lib/python2.4/httplib.py", line 810, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.4/httplib.py", line 833, in _send_request self.endheaders() File "/usr/lib/python2.4/httplib.py", line 804, in endheaders self._send_output() File "/usr/lib/python2.4/httplib.py", line 685, in _send_output self.send(msg) File "/usr/lib/python2.4/httplib.py", line 652, in send self.connect() File "/usr/lib/python2.4/httplib.py", line 636, in connect raise socket.error, msg socket.error: (99, 'Cannot assign requested address')

I am using Python 2.4.3. I do conn.close() too. But why is this error indicated?

+2
source share
3 answers

This is not a python problem.

In linux kernel 2.4, the range of ephemeral ports is from 32768 to 61000. Thus, the number of available ports = 61000-32768 + 1 = 28233. From what I understand, because the web service in question is pretty fast (<5ms actually), so Thus, all ports get used. The program should wait about a minute or two to close the ports.

What I did was count the amount of conn.close() . When the number was 28000, wait 90 seconds and reset the counter.

+8
source

BIGYaN correctly identified the problem, and you can make sure that by calling "netstat -tn" immediately after the exception occurs. You will see a lot of connections with the state "TIME_WAIT".

An alternative to waiting for port numbers to appear is to simply use a single connection for all requests. You are not required to call conn.close() after each call to conn.request() . You can simply leave the connection open until your requests are completed.

+5
source

I also came across a similar problem when executing several POST instructions using the python request library in Spark. To make matters worse, I used multiprocessing for each artist to send to the server. Thus, thousands of connections created in seconds took several seconds to change state from TIME_WAIT and free up ports for the next set of connections.

Of all the available solutions available on the Internet that talk about keeping keep-alive disabled using with request.Session () and others, I found this answer working, which uses the 'Connection': 'close' configuration as a header parameter. You may need to put the contents of the header on the separte line outside the command line.

 headers = { 'Connection': 'close' } with requests.Session() as session: response = session.post('https://xx.xxx.xxx.x/xxxxxx/x', headers=headers, files=files, verify=False) results = response.json() print results

This is my answer to a similar problem using the above solution.

+1
source

More articles:


All Articles