How to create a software load balancer for socket connections in java

Question

I plan to create a software load balancer that will sit in front of many socket servers on Linux. Clients connect to a load balancer. The load balancer will maintain a table of alternative ips and their ports. It will connect the client to the best available ip and port and disconnect from the client. Thus, it will exit the scene (will no longer be connected to the client). In this state, only the connected devices will be the client, and the new socket server, but NOT the load balancer.

Example : Cleint ip 10.1.2.3 port 1234 load balancer Ip 10.1.2.4 port 1235 list of socket servers in Load Balancer: A Ip 10.1.2.4 port 1236 B Ip 10.1.2.4 port 1237 C Ip 10.1.2.5 port 1238 Now for the 1st request to the load balancer from client, the load balancer will establish a connection between the client & server A and disconnect itself from client. for the 2nd request to the load balancer from client, the load balancer will establish a connection between the client & server B and disconnect itself from client. for the 3rd request to the load balancer from client, the load balancer will establish a connection between the client & server C and disconnect itself from client. Any Help on implementing this in Java is greatly appreciated. 

Answer 1

This miniature design can be useful for applications on small devices such as mobile phones and tabs. The limit set by the server on how many round-trip flights can be allowed from a specific device can be set. The extraordinary scaling of online servers will help reduce the number of round-trip flights.

Answer 2

I would use redis to store the lookup table. Each load balancer will search the redis lookup table to connect to the most accessible / highest priority server. This search will return a single integer, which is the server index. Each application in the client will store the ip server with its corresponding indexes. therefore, this search is very fast and less than 30 ms. No redirection required. Rollback is also provided in case of simultaneous connections, and qouta on the desired server ends by the time the application tries to connect to the search server. In this case, it will again search for the most accessible server, i.e. Start more recursively until it is successfully connected or all resources are completed and the connection request is marked as a dead end.

How about reserving a connection for a few milliseconds for each search? After the connection delay for this search has expired, the file page will be available for use for a new connection. This will reduce the recursive search, but will also block the connection. The delay should be enough to establish a connection, which may vary. On the other hand, new connections will be blocked during this delay, which can lead to poor user experience. You need a compromise between the two: reduce the search and block the connection, so as never to block the connection and endure a recursive search, which is very fast.

Answer 3

The difficulty lies in disconnecting the device from the server side and connecting it to the intended server.
There might be work around: use a redirect instead.

• Each server is a load balancer and service provider for client devices
• Each server will always keep track of its open file limits and a safe supply of defender from the maximum open file limits
• This pool will be used to verify the need for redirection.
• When the open file limit reaches the safe limit, any further connection request will return the next available ip server to the client device.
• The next closest available ip server can be supported in the lookup table in memory.
• The device checks to see if the return value is started with ip forwarding. It will automatically connect to the received ip. Otherwise, suppose that the device received the service from any server successfully.

Thus, we can avoid the limitations of the open file and connection failure.

Answer 4

I do not quite understand the requirement that the load balancer disconnect from the client. If your sockets are actually TCP connections, as they seem, I don’t see how you could offload a connection with a client working somewhere else without low-level hacking. For example, look at ldirectord from a linux virtual server . This allows you to completely unload the connection.

For pure simplicity, I just used HAProxy . Does most of what you want, with the exception of unloading the connection.

Finally, you can also use some kind of circular DNS solution. This will also disconnect the connection as necessary.