score:2

Accepted answer

For vertical solution: you can union many DataFrames

val dfs = x.map(field => spark.sql(s"select '$field' as fieldName, sum($field) from dftable"))
val withSum = dfs.reduce((x, y) => x.union(y)).distinct()

Probably this would be helpful

val sums = x.map(y => s"sum($y)").mkString(", ")
spark.sql(s"select $sums from dftable");

Sums will be in format: "sum(field1), sum(field2)"

You can use DSL also:

import org.apache.spark.sql.functions._
val sums = for (field <- x) yield { sum(col(field)) }
df.agg(sums : _*)

The result should be exactly the same

score:1

I hope this is helpful 

import org.apache.spark.sql.functions._
import spark.implicits._
val df1 = Seq((1,2,3), (3,4,5), (1,2,4)).toDF("A", "B", "C")
df1.describe().show()


val exprs = df1.columns.map(c => sum(col(c))).toList

df1.agg(lit(1).alias("temp"),exprs: _*).drop("temp")

Output:

+------+------+------+
|sum(A)|sum(B)|sum(C)|
+------+------+------+
|     5|     8|    12|
+------+------+------+

Related Query

More Query from same tag