score:1

you can use a udf to turn the start and end years into an array of all the years in the range, and then use explode to turn each item in that array into a row:

import org.apache.spark.sql.functions._
val torange = udf { (y1: int, y2: int) => (y1 to y2).toarray }

input
  .withcolumn("years", torange($"start_year", $"end_year"))
  .select($"name", explode($"years") as "year")

Related Query

More Query from same tag