score:1
Accepted answer
you can use concat_ws
function to create a string from an array as
val finaldf = df.withcolumn("_2", concat_ws(",", $"_2"))
.withcolumn("_4", concat_ws(",", $"_4"))
.withcolumn("_6", concat_ws(",", $"_6"))
hope this helps!
Source: stackoverflow.com
Related Query
- Converting WrappedArray to string in DataFrame
- Converting String to Date in Spark Dataframe
- Converting an Array[Double] Column into a string or two different columns with Spark Dataframe Scala
- Spark(scala): converting a JSON string to dataframe
- Filter spark DataFrame on string contains
- how to convert json string to dataframe on spark
- Scala safe way of converting String to Enumeration value
- Spark dataframe get column value into a string variable
- Converting JSON string to a JSON object in Scala
- How do I convert a WrappedArray column in spark dataframe to Strings?
- Aggregate rows of Spark DataFrame to String after groupby
- Extract words from a string column in spark dataframe
- How to add a column to Dataset without converting from a DataFrame and accessing it?
- Converting JsValue to String
- How to set column names to toDF() function in spark dataframe using a string array?
- Converting multiple different columns to Map column with Spark Dataframe scala
- How do I get a spark dataframe to print it's explain plan to a string
- How can I print nulls when converting a dataframe to json in Spark
- Converting empty string to a None in Scala
- How to transform Spark Dataframe columns to a single column of a string array
- Scala Spark : How to create a RDD from a list of string and convert to DataFrame
- How to read in-memory JSON string into Spark DataFrame
- Spark DataFrame handing empty String in OneHotEncoder
- Creating a Spark DataFrame from a single string
- Spark DataFrame not respecting schema and considering everything as String
- Filter dataframe on non-empty WrappedArray
- Converting a vector column in a dataframe back into an array column
- Converting dataframe columns into list of tuples
- spark scala : Convert DataFrame OR Dataset to single comma separated string
- How to get max length of string column from dataframe using scala?
More Query from same tag
- Why is there a sbt exception there, when I have added the folder with another git branch as a subfolder of the project?
- How to combine two RDD[String]s index-wise?
- Are there any reasons to use Action.async other than readability in Play Framework?
- Scala - Why does my for loop not return?
- How to pass extential type of tuple to invariant generic function?
- Spark worker nodes timeout
- Can I use a class variable in a Scala match statement?
- What are the compiler generated implicit values in Scala 2.10?
- Why intellij import project from SBT failed
- Sort array in Scala partially
- Where do I find and install the template editor for ScalaIDE?
- Creating Dataframe which uses BigInteger throws scala.MatchError: class java.math.BigInteger (of class java.lang.Class)
- Counting elements in array column with map
- intellij idea 13.1.4 update scala plugin to v0.41 cause permission denied
- Scala Akka Type unspecified value parameters
- Clone/Deep-Copy a Spark DataFrame
- Why does the `is not a member of` error come while creating a list in scala using the :: operator
- Slick 3.1, Left Joins and Filters
- type parameters and numeric widening
- How to change destination folder of compiled assets in PlayFramework 2.3.1?
- How to write to a csv file in scala?
- Algorithm mixing
- Scala: idiomatic way to merge list of maps with the greatest value of each key?
- How to get rid of extra escape characters while doing json transformation using scala play framework?
- Json.createBuilderFactory and createObjectBuilder hang
- How to present static file results that are generated at different time to client in play 2?
- Akka TypedActor - how to correctly handle async responses with context
- Calling Dataset static method does not work in Spark Shell
- Scala Removing Generic Type Information
- High order functions in Scala