Geek Answers Handbook

How to get a product from two RDD?

New to the spark. I have two RDDs and you want to create a referenced RDD as shown below.

val rdd1 =  Array(1, 2)
val rdd2 =  Array(a, b, c)

val resultRDD = [(1,a), (1,b), (1,c), (2,a), (2,b), (2,c)]

Can someone help me on what kind of transformations or actions I need to use to create resultRDD as above. FYI, I write in scala.

EDIT

Thank. for me, as shown below, sparking works.

    val data = Array('a', 'b')
    val rdd1 = sc.parallelize(data)

    val data2 = Array(1, 2, 3)
    val rdd2 = sc.parallelize(data2)

    rdd1.cartesian(rdd2).foreach(println)
+4
source share
2 answers
def cartesian[U](other: RDD[U])(implicit arg0: ClassTag[U]): RDD[(T, U)]

Return the Cartesian product of this RDD and another, that is, the RDD of all pairs of elements (a, b), where a is in this and b is in the other.

Dock here

+8
source

Bright reflex work for me, as shown below.

    val data = Array('a', 'b')
    val rdd1 = sc.parallelize(data)

    val data2 = Array(1, 2, 3)
    val rdd2 = sc.parallelize(data2)

    rdd1.cartesian(rdd2).foreach(println)
0
source

More articles:


All Articles