Geek Answers Handbook

Spark.driver.extraClassPath Multiple Cans

I am trying to use Spark through Python to access (via JDBC) the PostGres database and the MSSQL database in the same session. In the spark-defaults.conf file, I can get one or the other to work, but not one or the other.

These two work independently:

spark.driver.extraClassPath /Users/myusername/spark-1.6.1-bin-hadoop2.4/lib/postgresql-9.4.1208.jre6.jar

spark.driver.extraClassPath /Users/myusername/spark-1.6.1-bin-hadoop2.4/lib/sqljdbc4.jar

I tried these three and none of them work (I get the error "there is no suitable driver"):

spark.driver.extraClassPath /Users/myusername/spark-1.6.1-bin-hadoop2.4/lib/

spark.driver.extraClassPath /Users/myusername/spark-1.6.1-bin-hadoop2.4/lib/postgresql-9.4.1208.jre6.jar sqljdbc4.jar

spark.driver.extraClassPath /Users/myusername/spark-1.6.1-bin-hadoop2.4/lib/postgresql-9.4.1208.jre6.jar /Users/myusername/spark-1.6.1-bin-hadoop2.4/lib/sqljdbc4.jar

Thanks in advance.

+4
source share
1 answer

If you want to use several cans, you need to tie them together. If you are using Linux, the chain operator is :on Windows ;.

For example, on Linux, yours extraClassPathwill be:

spark.driver.extraClassPath /Users/myusername/spark-1.6.1-bin-hadoop2.4/lib/postgresql-9.4.1208.jre6.jar:/Users/myusername/spark-1.6.1-bin-hadoop2.4/lib/sqljdbc4.jar

On Windows:

spark.driver.extraClassPath /Users/myusername/spark-1.6.1-bin-hadoop2.4/lib/postgresql-9.4.1208.jre6.jar;/Users/myusername/spark-1.6.1-bin-hadoop2.4/lib/sqljdbc4.jar
+6

More articles:


All Articles