Erlang Chat System Distribution

Question

I just finished Erlang in Practice screencasts (code here ) and you have some distribution questions.

Here is the general architecture:

This is what the observation tree looks like:

Reading Distributed Applications makes me think that one of the main motives is switching to another resource.

However, is it possible, for example, the Message Router supervisor and its workers to be on one node, and the rest of the system is on another, without any code changes?

Or should there be 3 different OTP applications?

Also, how can this system scale horizontally? For example, if now I understand that my system can handle 100 users, and that I have defined Message Router as the main bottleneck, how can I just add another node, where now it can handle 200 users?

Answer 1

I developed Erlang applications only during my studies, but as a rule, we had many small processes that did just one thing and sent messages to other processes. And the beauty of Erlang is that it does not matter if you send a message within the same Erlang virtual machine or with the same Computer, the same local area network or via the Internet, the call and a pointer to another process are always the same for the developer.

So, you really want to have one application for each small part of the system.

It does not simplify the creation of an application that can scale. The rule of thumb states that if you want the application to run 10 times as many nodes, you need to rewrite it, because otherwise the service data for messaging would be too large. And, obviously, when you start from 1 to 2, you also need to consider this.

So, if you find a bottleneck, an application that is especially slow when processing too many clients, you want to run it a second time, and you need to perform additional additional load balancing, before the start of the second application.

Assume that the supervisor checks the contents of the message for inappropriate content and therefore is slow. In this case, the node everyone is talking to will be a simple router application that will redirect messages to different instances of the supervisor application using a round-robin roll. In case these 1 or 2 instances are not enough, you could write a router so that you can manipulate the number of instances by sending control messages.

However, for this to work automatically, you will need to conduct a different process for monitoring the servers and discover that they are overloaded or in use.

I know that dynamically adding and removing resources always sounds great when you hear about it, but as you can see, this is a lot of work, and you need to create a message system that allows this, as well as a monitoring system that can control the need.

I hope this gives you some insight into how this can be done, unfortunately, more than a year has passed since the writing of my last Erlang application, and I did not want to provide code that might be incorrect.