Geek Answers Handbook

Access to the Spark Web web interface from another place, except where the work is done

I have a spark cluster with 1 master 9nodes. I work offline. I do not have access to the web browser from any of the nodes in the cluster (I connect to the nodes via ssh -it, this is a grid5000 cluster). I was wondering if it is possible to access the Spark Web UI in this case? I tried to copy the logs from my cluster to SPARK_PATH / work on my local computer (leaving the impression that the tasks that were running in the cluster were running on my local machine). This idea came about after reading this part from the documentation:

If an application has logged events during its life cycle, the stand-alone web user interface automatically updates the application interface after the application terminates.

But that did not work. What I see in the user interface:

Applications: 0 Launch, 0 Completed Drivers: 0 Launch, 0 Completed Status: ALIVE

Thank!

+4
source share
1 answer

You do not need to copy anything, just go to port 8080 on the master machine or port 4040 on the application machine (while the application is running). If the machines are not accessible from the outside, you need to go through SSH.

SSH - , . ssh -D 9999, localhost:9999 -, FoxyProxy, Firefox, Chrome.

+3

More articles:


All Articles