Accepted answer

Chaining multiple when expressions on id column values:

val ids = Seq(1, 3, 4, 7)

val scoreCol = ids.foldLeft(lit(null)) { case (acc, id) =>
  when(col("sourceId")===id, col(s"score_$id")).otherwise(acc)

val df2 = df.withColumn("score", scoreCol)

Or building a map expression from score_* columns and use it to get score values:

val scoreMap = map(
    .flatMap(c => Seq(lit(c.split("_")(1)), col(c))): _*

val df2 = df.withColumn("score", scoreMap(col("sourceId")))

Related Query

More Query from same tag