score:2
the cleanest way i've seen so far is something along the lines of filtering out the first row
csv_rows = sc.textfile('path_to_csv')
skipable_first_row = csv_rows.first()
useful_csv_rows = csv_rows.filter(row => row != skipable_first_row)
Source: stackoverflow.com
Related Query
- Scala - Remove first row of Spark DataFrame
- Drop first row of Spark DataFrame
- Spark - Scala - Remove Columns from a dataframe based on condition
- Transpose DataFrame single row to column in Spark with scala
- Remove Null from Array Columns in Dataframe in Scala with Spark (1.6)
- Adding a list to a dataframe in Scala / Spark such that each element is added to a separate row
- Add a row to a empty dataframe using spark scala
- how to get the row corresponding to the minimum value of some column in spark scala dataframe
- how add a row to dataframe spark scala with different schema?
- Scala - First quartile, third quartile, and IQR from spark SQLContext dataframe without Hive
- Getting values of Fields of a Row of DataFrame - Spark Scala
- unable to remove trailing space in spark scala dataframe
- How to Remove first few lines/header from multiple files using scala in spark
- Iterate the row in dataframe based on the column values in spark scala
- Count of values in a row in spark dataframe using scala
- How to create a column of row id in Spark dataframe for each distinct column value using Scala
- Need to remove string "_value" in spark dataframe column names using scala
- Remove empty lists in Scala from ArrayType column in Spark Dataframe
- change a dataframe row value with dynamic number of columns spark scala
- Split single row into two rows in a spark dataframe using scala
- Get first not null value spark scala on a dataframe during aggregation
- How to add the incremental date value with respect to first row value in spark dataframe
- Calculating column value in current row of Spark Dataframe based on the calculated value of a different column in previous row using Scala
- Spark - Scala Remove special character from the beginning and end from columns in a dataframe
- How to fill column with value taken from a (non-adjacent) previous row without natural partitioning key using Spark Scala DataFrame
- Imputing null values in spark dataframe, based on the row category, by fetching the values from another dataframe in Scala
- How to load complex xml files containing more than 1 row tag into dataframe using spark scala and save it as table(note generic solution)
- Rdd verify Date format and remove row if date format is incorrect Scala Spark
- how to write a Scala program to remove a row from a dataframe if it has same value as some other row for last column column?
- Scala Spark Convert Dataframe and get all Unique IDs and its type from each row
More Query from same tag
- Scala: Can a generic trait take its generic type instance from its implementing class?
- Spark Streaming Kinesis consumer return empty data
- String cannot be cast to Integer(Scala)
- "Class org.apache.hadoop.fs.s3a.S3AFileSystem not found" on fatjar with sbt assembly
- Define Map with generic datatype value in Scala
- What can I use scala.Singleton for?
- How to send actor a tell message when a URL is accessed?
- Unbound Wildcard Type
- post and pre increment not working as expected for REPL variable in scala
- Secondary sorting by using join in Spark?
- How to use connection pool to postgresql in spark
- How is it that a {} block can represent a function in scala?
- how to get current application instance in companion object
- Can't submit job with Flink 1.5 cluster
- Scala: Iterate an Array's indices in for-comprehension but ensure result type is same Array type
- Replace a value in a row with a value in another row in Spark dataframe
- Existential Higher Kinded Method Parameter
- List[(Int,Int)] Into List[Int] - Seeking Advice
- Function Composition in Scala
- error Using bcc with play mailer and scala: addBcc is not a member of play.api.libs.mailer.Email
- Creating a Vaadin-sbt project
- How to load and process multiple csv files from a DBFS directory with Spark
- How to couple first element of array with every other element in it to obtain a list of tuples?
- Collecting dependencies under sbt 0.10 (putting all dependency jars to target/scala-version/lib/)
- What does Action mean in play framework (Scala)?
- Getting error like need struct type but got string in spark scala for simple struct type
- Import does not bring implicits in scope
- Scala type inference confusion
- Why does List[String] is matched for List[Int] in my example though I pass Manifest?
- Synchronization of Actor messages