Why does my Perl script TCP server support many TCP connections?

Question

I have a strange problem with a server that accepts TCP connections. Despite the fact that some processes are usually expected, it hangs with some volumes of connections.

Long version:

The server is written in Perl and associates the $srv socket with the reuse flag and listens == 5. Then it forks 10 processes with the cycle $clt=$srv->accept(); do_processing($clt); $clt->shutdown(2); $clt=$srv->accept(); do_processing($clt); $clt->shutdown(2);

A client written in C is also very simple - it sends some lines, then receives all available lines and does shutdown(sockfd, 2); . Nothing happens there, and at the end, both the send and receive queues are empty (as reported by netstat ).

The connection lasts only ~ 20 ms. All clients behave the same, are the same implementation, etc. Now let me say that I accept X connections from client 1 and another X from client 2. Processes still report that they are idle all the time. If I add another X connection from client 3, all of a sudden server processes will start hanging right after acceptance. The first lock they perform after accept(); , is while (<$clt>) ... , but they do not receive any data (from the first attempt already). Suddenly, all 10 processes are in this state and do not stop waiting. In strace server processes seem to hang on read() , which makes sense.

In the TIME_WAIT state owned by this server, there are many connections (~ 100 when the problem begins to appear), but it may be a red herring.

What could be here?

---

After another analysis: it turned out that the client was to blame, and did not close the previous connections properly before trying to perform the next one. The servers at the top of the load balancer list were abandoned by legacy connections.

Answer 1

This is probably not the solution to your problem, but it can solve the problem that you will encounter in the future: remember to close () the sockets when you are done! shutdown () will disable the stream, but it will still have a file descriptor.

Since you said strace shows processes stuck in read (), your problem is that the client is not sending the data that you expect from the send. You must either fix your client or add an alarm () to the server processes so that they can survive dead clients.

Answer 2

Is it a surge, and then a pause for a long time (about two minutes or so), and then a surge again? If so, you may not have the maximum maximum open maximum size of your system.