Geek Answers Handbook

Launch one Spark Slave (or Worker)

When i do it

spark-1.3.0-bin-hadoop2.4% sbin/start-slave.sh

I get this message

failed to launch org.apache.spark.deploy.worker.Worker:
                         Default is conf/spark-defaults.conf.

Even if I have this:

spark-1.3.0-bin-hadoop2.4% ll conf | grep spark-defaults.conf
-rw-rwxr--+ 1 xxxx.xxxxx ama-unix  507 Apr 29 07:09 spark-defaults.conf
-rw-rwxr--+ 1 xxxx.xxxxx ama-unix  507 Apr 13 12:06 spark-defaults.conf.template

Any idea why?

thank

+4
source share
2 answers

First of all, you have to make sure that you use this command correctly,

Usage: start-slave.sh <worker#> <spark-master-URL>

where <worker#>is the work number that you want to run on the computer on which this script is running.
<spark-master-URL>looks likespark://localhost:7077

+4
source

I use spark 1.6.1, and you no longer need to specify the worker number, so the actual use:

start-slave.sh spark://<master>:<port>
+6
source

More articles:


All Articles