Socket usage error when reusing sockets

Question

I am writing an XMLRPC client in C ++ that is designed to communicate with an XMLRPC server in python.

Unfortunately, at the moment, python-based XMLRPC server is only able to issue one connection request, then it disconnects, I discovered this thanks to mhawke's answer to my previous request about a related subject

Because of this, I need to create a new socket connection to my python server every time I want to make an XMLRPC request. This means creating and removing multiple sockets. Everything is working fine until I approach ~ 4000 queries. At this point, I get socket error 10048, Socket in use .

I tried a sleep thread so that winsock corrected its file descriptors, a trick that worked when my python client had the same problem, but to no avail. I tried the following

int err = setsockopt(s_,SOL_SOCKET,SO_REUSEADDR,(char*)TRUE,sizeof(BOOL)); 

without success.

I use winsock 2.0, so WSADATA :: iMaxSockets should not enter the game, and in any case, I checked its set to 0 (I assume it means infinity)

4,000 requests do not seem like the wild number of requests that need to be made at application startup. Is there a way to use SO_KEEPALIVE on the client side while the server is constantly shutting down and reopening?

Am I missing something at all?

Answer 1

The problem arises because of the freezing of sockets in the TIME_WAIT state, which is entered after the client socket is closed. By default, the socket will remain in this state for 4 minutes before it is available for reuse. Your client (may have helped other processes) consume them all for a 4-minute period. See this answer for a good explanation and a possible solution without code.

Windows dynamically allocates port numbers in the range 1024-5000 (3977 ports), unless you explicitly bind the socket address. This Python code demonstrates the problem:

 import socket sockets = [] while True: s = socket.socket() s.connect(('some_host', 80)) sockets.append(s.getsockname()) s.close() print len(sockets) sockets.sort() print "Lowest port: ", sockets[0][1], " Highest port: ", sockets[-1][1] # on Windows you should see something like this... 3960 Lowest port: 1025 Highest port: 5000 

If you try to perform this operation again, this will happen very quickly, since all dynamic ports are in TIME_WAIT state.

There are several ways:

• Manage your own port assignments and use bind() so that the client socket for a specific port that you increase create a socket. You will still have to handle the case when the port is already in use, but you will not be limited to dynamic ports. eg.

   port = 5000 while True: s = socket.socket() s.bind(('your_host', port)) s.connect(('some_host', 80)) s.close() port += 1 

• Script with socket SO_LINGER option. I found that this sometimes works on Windows (although it is not entirely clear why): s.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, 1)

• I do not know if this will help your specific application, however, you can send multiple XMLRPC requests over the same connection using multicall . This basically allows you to accumulate multiple requests and then send them all together. You will not get any answers until you really send the accumulated requests, so that you can essentially think of it as a batch processing - does this fit your application design?

Answer 2

Update:

I threw this in code and it seems to be working now.

 if(::connect(s_, (sockaddr *) &addr, sizeof(sockaddr))) { int err = WSAGetLastError(); if(err == 10048) //if socket in user error, force kill and reopen socket { closesocket(s_); WSACleanup(); WSADATA info; WSAStartup(MAKEWORD(2,0), &info); s_ = socket(AF_INET,SOCK_STREAM,0); setsockopt(s_,SOL_SOCKET,SO_REUSEADDR,(char*)&x,sizeof(BOOL)); } } 

Basically, if you encounter error 10048 (using a socket), you can just close the socket, clear the call and restart WSA, reset the socket and its sockopt

(last sockopt might not be needed)

I must have not seen WSACleanup / WSAStartup calls before because closesocket () and socket () were definitely called

this error only occurs with every 4000ish call.

I am curious why this might be, although this seems to fix it. If anyone has any data on this, I would be very interested to hear it.

Answer 3

Do you close sockets after using it?