score:2
Accepted answer
you can use either of the two:
employeedf.na.fill(1000, seq("commission")).show()
import org.apache.spark.sql.functions._
employeedf.withcolumn("commission", when(col("commission").isnull, 1000) otherwise col("commission")).show()
Source: stackoverflow.com
Related Query
- Not able to replace null values in Spark Dataframe with Integer value
- Replace Null Values of a Column with mean of another Categorcial Column in Spark Dataframe
- Tried/failed to replace null values with means in spark dataframe
- Replace Not null values of Spark dataframe as “1” using Scala in optimized way
- Spark Scala: How to replace null with values from an array or another dataframe
- Spark dataframe not able to Compare Null values
- How to replace null values with above/below not null value on same column in Data-frame using spark?
- Replacing null values with 0 after spark dataframe left outer join
- Replace missing values with mean - Spark Dataframe
- Replace null values in Spark DataFrame
- Spark UDF not working with null values in Double field
- Join in spark dataframe (scala) based on not null values
- Replace Empty values with nulls in Spark Dataframe
- How to replace null NAN or Infinite values to default value in Spark Scala
- Fill null values in dataframe column with next value
- Replace null values in a list with another value in Scala
- Spark: replace null values in dataframe with mean of column
- Add a column to spark dataframe which contains list of all column names of the current row whose value is not null
- replace null value in the column of a dataframe with the value in other dataframe wrt to id
- Insert NULL values into Hive with dataFrame in Spark
- Spark dataframe replace values of specific columns in a row with Nulls
- Replace spark dataframe column value with random value (ex. UUID)
- How to replace Map with null if map key from (key,struct) is null in spark dataframe
- Get first not null value spark scala on a dataframe during aggregation
- Find average value from a column of stream dataframe with array values using spark scala
- Replace randomly RDD values to null with scala spark
- Spark dataframe - Replace tokens of a common string with column values for each row using scala
- Scala: Transform and replace values of Spark DataFrame with nested json structure
- select distinct(On one column) not null (on all other column) value from Dataframe in apache spark
- Replace null in a column of a dataframe with other value
More Query from same tag
- Mongodb Scala Driver: How to test an error?
- How to combine multiple TIFF's into one large Geotiff in scala?
- Capturing all occurrences of interleavened nodes with StandardTokenParsers in Scala
- Scala: Return multiple data types from function
- Failed to execute user defined function on a dataframe in Spark (Scala)
- FS2 (Functional Streams for Scala) how to do group(n)
- writing a custom get method for a scala map
- What mechanism use an Akka Actor to persist its messages?
- What is a good way to implement a "delayed" list stream in Scalaz
- Play Json add field which is not present in class
- How does one "override" an inner class in Scala?
- Jetty works for HTTP but not HTTPS
- How to access type alias of generic type in Scala?
- Not able to translate Stream(BSONDocument(<non-empty>) to a BSONDocument
- In Scala, how to instantiate a class N times in another class
- Domain routes in Play Framework 2.3
- Play Framework won't run inline javascript
- Scala Function vs Class Dichotomy
- What is the FunctionN trait that represents a function taking a by-name parameter?
- How to use length function in lpad, where the length of lpad is decided based on the column value
- How to use custom plugin in multi-module project?
- How to create a TestActorRef inside a test class for an Actor with constructor params?
- Enable JDK 7 in Eclipse Indigo on Mountain Lion
- Performance difference in toString.map and toString.toArray.map
- Designing referentially transparent function for reading from channel
- Scala case class with cached hashCode
- Play Framework json labels without quotes
- How do I "get" a Scala case object from Java?
- How does spark report / collect metrics
- Scala: How to know if a class is an enumeration; isInstanceOf[Enumeration] doesn't work