SparkSQL reads from a MySQL database table using Python

Question

MySQL has a "user" table. I want to read it in my Spark SQL program. How can I read a table from MySQL in Apache Spark SparkSQL using Python? Is there a connector that I can use for this task? Thanks.

Answer 1

There is a similar question . Start pyspark as follows

 ./bin/pyspark --packages mysql:mysql-connector-java:5.1.38 

Then just run

 sqlContext.read.format("jdbc").options( url ="jdbc:mysql://localhost/mysql", driver="com.mysql.jdbc.Driver", dbtable="user", user="root", password="" ).load().take(10) 

It most likely will just work. But it depends on your mysql setting, so if it does not try to change the password, username, db-url and other parameters.