Increase outgoing connections on CentOS

Question

I learned from this article: Scaling up to 12 million concurrent connections: like MigratoryData Did It , that it is possible to make more than 64K connections from one client with multiple IP addresses.

Now I have an AW2 ec2 machine that has 10 IPs for testing. Configuration in /etc/sysctl.conf

fs.nr_open = 2000000
fs.file-max = 2000000

And config in /etc/security/limits.d/def.conf

*       soft    nofile  2000000
*       hard    nofile  2000000

I start one process (written in C) and create 60,000 connections from the first IP address. Everything is working fine. Then I started another process and tried to create 60,000 connections from the second IP address, but it gets an error when the number of connections reaches about 7500 (total: 67500). The error message Connection timed out.

The problem is not the file descriptor limiter, as I can still open / read / write files on the client machine. But any output connection to any remote server will expire.

The problem is not the server side, because the server can accept many more connections from different client machines.

It seems that there are some settings, not the number of open files that limit the number of outgoing connections. Can anyone help?

Answer 1

To be able to open more than 65536 TCP socket connections from your client computer, you really need to use more IP addresses.

Then, for each TCP socket connection, you must tell the kernel which IP address and which ephemeral port to use.

, TCP , , TCP IP-, , .

MigratoryData Benchmark Tools Java, , TCP- . , ++.

, TCP- 192.168.1.1:8800 , 192.168.1.10 IP- , IP- 192.168.1.10 - , 12345 - IP- 192.168.1.1 8800, - :

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include<sys/socket.h> 
#include <arpa/inet.h>

int main(int argc, char *argv[])
{
    int n, sockfd;
    char buffer[1024];
    struct sockaddr_in localaddr, remoteaddr;


    sockfd = socket(AF_INET, SOCK_STREAM, 0);

    localaddr.sin_family = AF_INET;
    localaddr.sin_addr.s_addr = inet_addr("192.168.1.10");
    localaddr.sin_port = htons(12345); 
    bind(sockfd, (struct sockaddr *) &localaddr, sizeof(localaddr));

    remoteaddr.sin_family = AF_INET;
    remoteaddr.sin_addr.s_addr = inet_addr("192.168.1.1");
    remoteaddr.sin_port = htons(80);
    connect(sockfd, (struct sockaddr *) &remoteaddr, sizeof(remoteaddr));

    n = read(sockfd, buffer, 512);

    // ...

    close(sockfd);

    return 0;
}