Erlang language does not expose topics, it gives you Erlang processes. These processes are planned efficiently by Erlang efficiently on OS threads, which are typically mapped to CPU cores. They are light weight (less than 4KB of memory on a 32-bit virtual machine, including the initial heap) and are planned proactively so that blocking or high processor consumption in any of them does not deny any other process a fair share of the processor time.
So don’t be afraid to spawn a process for processing each of the requests that you want to serve on your system - this is a good initial design and usually gives good bandwidth using parallelism and tends to scale to more cores / cpus / nodes is easier.
An additional advantage is that the code in each process can be written in a simple way:
%% Ask a server to perform a request and await the response from the worker. request(Server, R) -> Server ! {new_request, R, self()}, receive {response, Response} -> Response end. %% Create a server. start() -> spawn(?MODULE, server, []). %% The server code server() -> receive {new_request, R, Sender} -> %% Spawn a process to handle this request spawn(?MODULE, process_request, [R, Sender]), server() end. %% The worker code process_request(R, Sender) -> A = do_io(), B = do_cpu_bound_thing(A), C = do_io(C), Sender ! {response, C}. % Return the response to the sender %% Process shuts down cleanly here as there nothing more to do.
Here we have two types of processes: a single central server process that accepts new requests and any number of work processes that actually do the work. Errors in individual requests do not affect the server process or other workflows; individual workflows may run at different speeds depending on the I / O resources and CPU.
From here it’s easy to add control over workflows so that we can restart individual requests that fail, distributed processing with several machines, adding the “Node” argument to the spawn call to create workers, timeouts, so that clients make requests are not blocked forever, if the server is overloaded or the workflow is not working, etc.
You cannot get the parallelism of the above code using multiple handlers inside the gen_event process. The gen_event code will be more confusing to read, and you will have to alternate with the requests yourself, rather than let the executables do this for you.
tl; dr: the overhead is so low and the other benefits are so great that you usually (almost always) spawn a process, rather than try and perform several actions in the gen_event process at once.