Spark Clusters: Employee Information Not Displayed in Web UI

Question

I have installed spark isolation on many clusters. And I tried to start the clusters through a cluster start script. I added the cluster IP address to the conf / slaves file. The wizard connects to all slaves via ssh without a password. After running the ./bin/start-slaves.sh script, I get the following message:

launch org.apache.spark.deploy.worker.Worker, log in to / root / spark -0.8.0-incubating / bin /../ logs / spark-root-org.apache.spark.deploy.worker. Worker-1-jbosstest2.out

But the webUI of the wizard (localhost: 8080) does not show any information about the employee. But when I add the localhost entry to my conf / slaves file, the localhost workplace information is displayed.

There are no error messages, a message on the terminal says that the worker is running, but the WebUI does not show any workers.

Answer 1

I had the same problem. I noticed that I can not work with telnet: the port is from slaves. In my etc / hosts file (on the main one) I had the main entry 127.0.0.1 (before my 192.168.0.x wizard). When I deleted the 127.0.0.1 entry from my etc / hosts file, I could telnet, and when I started start.s (from the wizard), my subordinates connected

Answer 2

When you start the cluster, check the $jps in the work nodes, whether it is correct and check it in the logs using the worker PID.

or

install the following: start the cluster and check whether the ports with configured ports are inserted or not

 export SPARK_MASTER_WEBUI_PORT=5050 export SPARK_WORKER_WEBUI_PORT=4040 

Answer 3

check your / etc / hosts and see the bindings for the wizard

If your wizard is bound to localhost, as well as to the ip-address (for example, 192.168.xx), delete localhost. if you have a local host, an intact master will be mapped to localhost, which will prevent the slaves from connecting to the main IP address

Answer 4

You can use: ./start-master.sh --host 192.168.xx instead of modifying the file: /etc/hosts