What is the best way to communicate between two servers?

Question

I am creating a two-part web application. In one part, it uses a real-time connection between the server and the client, and in the other part, it performs some intensive processor task to provide relevant data.

Real-time communication implementation in nodejs and processor intensive part in python / java. How can a nodejs server participate in two-way communication with another server?

Answer 1

For the main solution, you can use Socket.IO , if you already use it and know how it works, it will do its job since it allows you to communicate between the client and server, where the client can be another server in another language.

If you need a more robust solution with additional options and controls, or that can handle higher traffic bandwidth (although this should not be a problem if you ultimately just send it through a relatively slow Internet), you can look at something like ØMQ ( ZeroMQ ). This is a message queue that gives you more control and many different communication methods besides request-response.

When you install both options, I would recommend using your server with an intensive processor as a stable end (server) and your web server as your client. Assuming you are using the same server for tasks with an intensive processor, and you are using multiple instances of the NodeJS server to use multi-core processors for your web server. This simplifies your communication, as you want to have one point to connect.

If you plan to use multiple CPU servers, you need to set up a routing server that can route between multiple web servers and multiple CPU servers, in which case I would recommend additional work on training ØMQ.

Answer 2

You can use the http.request method provided for curl request in node code. The http.request method is also used to implement api authentication. You can put your callback in the success of the request, and when you get the response data in node, you can send it back to the user. Although in backgrount, a java / python server can use a node request for an intensive processor task.

Answer 3

I support a node.js application that interacts between 34 tasks distributed across 2 servers.

In your case, for communication between the web server and the application server, you can consider mqtt.

I use mqtt for this kind of communication. Most languages ​​have mqtt clients, including node / javascript, python, and java. In my case, I publish json messages using mqtt 'themes, and any task registered to subscribe to the "theme" gets it when published. If you find "pub sub", "mqtt" and "mosquitto", you will find many links and examples. Mosquitto (now the Eclipse project) is just one of several mqtt brokers available. Another very good broker written in Java is called hivemq.

This is a very simple, reliable solution that scales well. In my case, literally millions of messages reliably go through mqtt every day.

Answer 4

You should look for socketio

Socket.IO enables real-time bidirectional communication based on events. It works on every platform, browser or device, equally focusing on reliability and speed.

Sockets have traditionally been the solution around which most real-time systems are archived, providing a bi-directional communication channel between the client and server.