Geek Answers Handbook

Spark-submit does not find the main class, although it exists in the jar file

First: here is the jar application file to send:

$ls -rlta /shared/ysgood/target/yardstick-spark-uber-0.0.1.jar -rw-r--r-- 1 steve staff 138611565 Aug 6 01:41 /shared/ysgood/target/yardstick-spark-uber-0.0.1.jar

Here is the class to send:

 01:55:02/ysgood $jar -tvf target/yardstick-spark-uber-0.0.1.jar | grep SparkCoreRDDBenchmark.class 15091 Thu Aug 06 01:36:30 PDT 2015 org/yardstick/spark/SparkCoreRDDBenchmark.class

Here is an attempt to introduce:

 $spark-submit --master $MASTER --class org.yardstick.spark.SparkCoreRDDBenchmark target/yardstick-spark-uber-0.0.1.jar NOTE: SPARK_PREPEND_CLASSES is set, placing locally compiled Spark classes ahead of assembly. Error: Cannot load main class from JAR file:/shared/ysgood/org.yardstick.spark.SparkCoreRDDBenchmark

Regarding the error: note that the path to the bank is incorrect: next

/shared/ysgood/org.yardstick.spark.SparkCoreRDDBenchmark

doesn't make sense: missing jar file path

target / measure-spark super-0.0.1.jar

+4
source share
1 answer

After the class, you should not put the package of your main class in the path to the main class. So check to see if your code has a SparkCoreRDDBenchmark class in the org.yardstick.spark package. If so, try to start the can without Spark, see if an error occurs. Unable to load main class. Maybe where problems arose when the bank was established. Good luck

0
source

More articles:


All Articles