Spark dataframe gets the column value in a string variable

Question

I am trying to extract a column value into a variable so that I can use the value somewhere else in the code. I am trying to do the following

 val name= test.filter(test("id").equalTo("200")).select("name").col("name")

He returns

 name org.apache.spark.sql.Column = name

how to get the value?

Answer 1

col("name")provides column type data. If you want to extract data from the "name" column, just do the same thing without col("name"):

val names = test.filter(test("id").equalTo("200"))
                .select("name")
                .collectAsList() // returns a List[Row]

Then for the string, you can get the name in String:

val name = row.getString(0)

Answer 2

val maxDate = spark.sql("select max(export_time) as export_time from  tier1_spend.cost_gcp_raw").first()

val rowValue = maxDate.get(0)