score:0
with your current implementation, you can just replace matrixentry
and add .collect.toseq
at the end of the process, like:
val value:rdd[string] = sc.textfile("file:///usr/local/data.csv")
val data: seq[string] =
sc.parallelize(value).map {
line => {
val fields = line.split(":")
val i = fields(0).tolong
val j = fields(1).tolong
val r = fields(2).todouble
"%d,%d,%f".format(i, j, r)
}
}.collect().toseq
Source: stackoverflow.com
Related Query
- How do I convert csv file to rdd
- How to read csv file and convert one column to Map[String, String] type in Scala+Spark?
- How to merge two text files and convert it to csv file in Scala
- How to convert a space delimited file to a CSV file in Scalar spark?
- How convert DataFrame to csv file and put it to remote SFTP server?
- How to convert hadoop avro, parquet, as well as text file to csv without spark
- how to convert nested json file into csv in scala
- How can I convert from a CSV file to scala toList?
- How to convert RDD[some case class] to csv file using scala?
- How to convert csv file in S3 bucket to RDD
- How to convert Array[String] into spark Dataframe to save CSV file format?
- How to convert csv file to Seq[String]?
- How to convert spark log files into one CSV File
- How do I read a large CSV file with Scala Stream class?
- In Scala, how to read a simple CSV file having a header in its first line?
- How to specify schema for CSV file without using Scala case class?
- Spark-SQL : How to read a TSV or CSV file into dataframe and apply a custom schema?
- How to load a CSV file into Apache Arrow vectors and save an arrow file to disk
- How to use Spark-Scala to download a CSV file from the web?
- How to read csv file into an Array of arrays in scala
- How to load the csv file into the Spark DataFrame with Array[Int]
- How to create a schema from CSV file and persist/save that schema to a file?
- How to create a graph from a CSV file using Graph.fromEdgeTuples in Spark Scala
- How can I read a CSV file and put its content in a Map in Scala?
- How to convert file with array of double to dataframe in spark?
- How to use NOT IN from a CSV file in Spark
- Scala: Convert xml dataframe to csv file
- How to convert Play! 2.0 Scala file into a format for CSVReader?
- How do I read a non standard csv file into dataframe with python or scala
- How to read a csv file as Map for key value pair
More Query from same tag
- Can't connect to Cassandra docker container in Scala Tests
- how to put the "<url>" to "</url>" into List?
- Scala get html content from web page
- Differents between single and singleOpt in Play framework and scala
- How do you declare a type alias in a scala constructor?
- Spark - Serializing an object with a non-serializable member
- context.become changes the behaviour of Akka Actor
- Why could streaming join of queries over kafka topics take so long?
- update the last element of List
- How to loop over a list of children found inside a single scala.xml.Node
- Using Scala reflection with Java reflection
- Spark - How to add a StructField at the beginning of a StructType in scala
- How to write idiomatic scala code for http4s
- What's the best way to create a dynamically growing array in Scala?
- define your own exceptions with overloaded constructors in scala
- In Akka, can I spoof a message from a different actor?
- Spark Persist Method is not getting invoked
- Is MongoDB-backed persistence still in Akka?
- Spray reverse proxy: keep transferring data after client has disconnected
- How do I specify a type bound for Float and Double on a generic type in Scala?
- Is `scalaBinaryVersion` incompatible with `crossScalaVersions` in SBT?
- Writing a function that returns => T
- What is non blocking and blocking future in Scala?
- Failed to update Gitlab commit status for project '120': HTTP 403 Forbidden
- Scalaz === on Some
- Spark: long delay between jobs
- Spark MLlib libsvm issues with data
- How to exclude transitive dependency in Sbt ( in context of assembly plugin )?
- How can I use MockitoSugar to mock an object (singleton) method?
- Appropriate return type for an async http request? IO[Future[Response]] or Future[Response]? Or some alternative?