Error starting spark wizard when slf4j is not installed

Question

I am using ubuntu to install a standalone spark. But he could not find the slf4j package while I use the pre-built spark.

./spark-1.4.1-bin-without-hadoop/sbin/start-master.sh Spark Command: /usr/lib/jvm/java-7-oracle//bin/java -cp /root/spark-1.4.1-bin-without-hadoop/sbin/../conf/:/root/spark-1.4.1-bin-without-hadoop/lib/spark-assembly-1.4.1-hadoop2.2.0.jar -Xms512m -Xmx512m -XX:MaxPermSize=256m org.apache.spark.deploy.master.Master --ip localhost --port 7077 --webui-port 8080 ======================================== Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/Logger at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2615) at java.lang.Class.getMethod0(Class.java:2856) at java.lang.Class.getMethod(Class.java:1668) at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486) Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 6 more 

Sorry I should not use spark-1.4.1-bin-without-hadoop, and it works when I load spark-1.4.1-bin-hadoop2.6. So this may be the problem of my use, not Spark.

Answer 1

An easy fix can use the class path from the pathto hasoop command, as suggested in the Spark Documentation :

 export SPARK_DIST_CLASSPATH=$(hadoop classpath)