Erlang Slave Module: Code and I / O Distribution

Question

We use Erlang with a slave module in an application that spawns slaves on different machines that communicate and are coordinated by a node master on another computer. Slave nodes open ports on their machines and run some external programs (in fact, erlang subordinate nodes (we call them workers) are just fancy wrappers around external programs).

However, we encountered some unexpected problems for which we did not find good solutions.

• Distribution of codes. Currently, our Makefile rsyncs compiled the erlang code (ebin folder) on the machines running the work nodes, and we load them with the -pa argument when the working node starts. There really should be some way to automatically distribute the code at runtime through Erlang, but we are not sure how to do this.

• Logging

  . The slave documentation says: "All TTY output created on the slave will be sent back to the master node." However, when we run lager (basho logger) on our subordinate (working) nodes, its output is not redirected to the main node tty (there is only log output from the node wizard). Currently, a process is running on the master node that logs (through lager) messages that it receives from subordinate nodes. Therefore, in order to register something on subordinate nodes, we send messages to the node master.

We start the work nodes as follows:

 slave:start(IP, NodeName, NodeArgs) 

where NodeArgs is

  -setcookie thecookie -pa /home/dau/mapro/deps/nicedecimal/ebin/ /home/dau/mapro/deps/meck/ebin/ /home/dau/mapro/deps/lager/ebin/ /home/dau/mapro/deps/jsx/ebin/ /home/dau/mapro/apps/worker/ebin/ /home/dau/mapro/ebin/ -s worker_app 

where all the paths indicated are absolute paths on the machines on which the working nodes are running.

Answer 1

You are doing the right thing with regard to point # 1: you are responsible for installing the source Erlang modules or precompiled beam files on the remote host computers, just as you would for installing OTP / Erlang itself. Once they are on the way to the code on remote computers, you can upload and download and upload the contents of your heart, though :)

wrt item # 2: have you confirmed that your slave code can output naive error messages (i.e. erlang:display() ) to your node master? in this case, the problem is in the lager configuration - try setting error_logger_redirect to false as a quick test, maybe ... sorry in advance if this is not useful!