score:10

Accepted answer

Indeed, the VectorAssembler Transformer does not take strings. So you need to make sure that your columns match numerical, boolean, vector types. Make sure that your udf is doing the right thing and be sure that none of the columns has StringType.

To convert a column in a Spark DataFrame to another type, make it simple and use the cast() DSL function like so:

val analysisData  = dataframe_mysql.withColumn("Event", dataframe_mysql("Event").cast(DoubleType))

It should work!


Related Query

More Query from same tag