How, if at all, do Erlang processes do in Nerner Threads?

Question

Erlang is known for supporting MANY light processes; he can do this because they are not processes in the traditional sense, or even threads, as in P-threads, but threads are completely in user space.

It’s good and good (really fantastic). But how then do Erlang threads execute in parallel in a multi-core / multi-processor environment? Of course, should they somehow map to kernel threads to run on individual kernels?

Assuming this is how it is done? Are many lightweight processes mapped to a single kernel thread?

Or is there another way to solve this problem?

Answer 1

The answer depends on the virtual machine used:

1) non-SMP: there is one scheduler (OS thread) that runs all Erlang processes taken from the pool of running processes (i.e. those that are not blocked, for example, receive )

2) SMP: There are K schedulers (OS threads, K are usually several processor cores) that runs Erlang processes from the common process queue. This is a simple FIFO queue (with locks for simultaneous access from multiple OS threads).

3) SMP in R13B and newer: there will be K schedulers (as before) that runs Erlang processes from several process queues . Each scheduler has its own queue, so the logic of process migration from one scheduler to another will be added. This solution will improve performance by avoiding excessive blocking in the queue of shared processes.

For more information, see this document by Kenneth Lundin, Ericsson AB, for the Erlang User Conference, Stockholm, November 13, 2008.

Answer 2

I want to change the previous answers.

Erlang, or rather, the Erlang runtime system (erts), by default sets the number of schedulers (OS threads) and the number of rows per number of processed items on your platform. These are processor cores or hardware threads. You can change these parameters at runtime using:

 erlang:system_flag(schedulers_online, NP) -> PrevNP 

Erlang processes do not yet have anything to do with any scheduler. Logical balancing of processes between schedulers is performed according to two rules. 1) A hungry scheduler will steal a job from another scheduler. 2) Migration paths are configured to push processes from schedulers with a large number of processes for schedulers at lower cost. This is done to ensure fairness in the number of cuts (lead time) for each process.

However, schedulers may be blocked for certain processing elements. This is not done by default. To enable erts to use the scheduler-> core change:

 erlang:system_flag(scheduler_bind_type, default_bind) -> PrevBind 

You can find several other types of bindings in the documentation. Using affinity can significantly improve performance under heavy loads! Especially in situations with a high degree of blockage. In addition, the Linux kernel cannot handle hyperthreads, to say the least. If you have hypertexts on your platform, you really should use this function in erlang.

Answer 3

I'm just guessing here, but I would suggest that there are a small number of threads that select processes from a common process pool for execution. When a process performs a blocking operation, the thread executing it snoozes it and selects another. When a running process forces another process to be unblocked, this newly unblocked process is put into the pool. I believe that a thread can also stop a process from running, even if it is not blocked at certain points to serve other processes.

Answer 4

I would like to add some input to what was described in the accepted answer.

Erlang Scheduler is an integral part of the Erlang Runtime system and provides its own abstraction and implementation of the concept of lightweight processes on top of OS threads.

Each scheduler runs on a single OS thread. As a rule, there are as many schedulers as the processor (cores) are on the equipment (it is configured and, of course, does not bring much significance when the number of schedulers exceeds the number of hardware cores). The system can also be configured so that the scheduler will not move between OS threads.

Now that the Erlang process is created, ERTS and Scheduler are solely responsible for managing the life cycle and resource consumption, as well as for memory, etc.

One of the main implementation details is that each process has a temporary reduction budget of 2000, available when the Scheduler takes this process from the execution queue. Every progress in the system (even I / O) is guaranteed to have a budget reduction. This is what makes pro-ERTS a multi-tasking system.

I would recommend an excellent blog post on this topic from Jesper Louis Andersen http://jlouisramblings.blogspot.com/2013/01/how-erlang-does-scheduling.html

As a short answer: Erlang processes are not OS threads and do not display them directly. Erlang schedulers are what run on OS threads and provide intelligent implementation of the finer Erlang processes that hide these details behind the eyes of the programmer.