score:1

Accepted answer

the main thing here is that generate a query list from given map that you can do like below

//input df
val df=seq(("john","non hf","new york"),("steav","non hf","mumbai"),("ram","hf","boston")).todf("srccolumnz", "srccolumny", "srccolumnr")

//input list

val maplist=list(map("targetcolumn" -> "columnnamex", "sourcecolumn" -> list("srccolumnx", "srccolumny", "srccolumnz", "srccolumnp", "srccolumnq", "srccolumnr")), map("targetcolumn" -> "columnnamey", "sourcecolumn" -> list("srccolumny")), map("targetcolumn" -> "columnnamez", "selectvalue" -> 5))

//get all the columns of df as list

val dfcols=df.columns.tolist

//then generate query list like below

val query = maplist.map { mp =>
            if (mp.contains("sourcecolumn")) {
                val srccolumn = mp.getorelse("sourcecolumn", "sourcecolumn key not found").tostring.replace("list(", "").replace(")", "").split(",").map(_.trim).tolist
                val srccol = srccolumn.filter(dfcols.contains(_)).head
                df.col(srccol.tostring).alias(mp.getorelse("targetcolumn", "no target column found").tostring)
            } else {
                lit(mp.getorelse("selectvalue", "no target column found").tostring.replace("(", "").replace(")", "").trim).alias(mp.getorelse("targetcolumn", "no target column found").tostring)
            }
        }

//finally , fire the query

df.select(query:_*).show

//sample output:

+-----------+-----------+-----------+
|columnnamex|columnnamey|columnnamez|
+-----------+-----------+-----------+
|     non hf|     non hf|          5|
|     non hf|     non hf|          5|
|         hf|         hf|          5|
+-----------+-----------+-----------+

Related Query