score:0
Accepted answer
try
var data = sc.textfile(mylist.mkstring(","))
alternatively, we can read each text file, then union
the resulting rdds:
import scala.util.{try, success}
val rdds = mylist.flatmap { f =>
try(sc.textfile(f)) match {
case success(rdd) => some(rdd)
case _ => none
}
}
val rdd = sc.union(rdds)
Source: stackoverflow.com
Related Query
- Passing a list of tuples as a parameter to a spark udf in scala
- Passing each element of a List to a function in Scala
- Passing the scala list element as a parameter in a filter expression
- Passing List as argument in spark scala function gives error
- Passing a list as varargs Scala
- Scala list concatenation, ::: vs ++
- Difference between a Seq and a List in Scala
- Appending an element to the end of a list in Scala
- Difference between Array and List in scala
- How to get Scala List from Java List?
- Preferred way to create a Scala list
- Scala how can I count the number of occurrences in a list
- How to convert a java.util.List to a Scala list
- How to sort a list in Scala by two fields?
- Scala check if element is present in a list
- Convert a Scala list to a tuple?
- How in Scala to find unique items in List
- Replace element in List with scala
- Finding the index of an element in a list scala
- Filtering a Scala List by type
- Using Scala from Java: passing functions as parameters
- Convert list in Scala to a formatted string
- Convert Java List to Scala Seq
- Scala - convert List of Lists into a single List: List[List[A]] to List[A]
- Spark Scala list folders in directory
- How to flatten a List of Futures in Scala
- Why are List and String identifiers named "xs" (in Scala and other languages)?
- How do you turn a Scala list into pairs?
- Add element to a list In Scala
- How to get methods list in scala
More Query from same tag
- Scala XML: create a node not using literals
- One simple spark program in scala : println out all the element in the RDD
- Query Parameters in case class - Swagger Docs
- type level pattern matching in scala
- Apache Spark: Importing jars
- Apply method used in type [Scala]
- Scalaz 7 - why using type alias results in ambigous typeclass resolution for Reader
- Same type after erasure in opaque types in Scala 3
- Typesafe Play sbt-plugin version 2.8.7 missing from maven repo1
- How to change a form's action in Lift
- transitive implicits - is this possible in Scala?
- One-off object pooling with Actor provider
- How to auto generate object when the class get no params
- Is there a straightforward and general way to wrap a scala object's code with code to be executed before and after the object body code?
- regexing a string that sometimes has a space and sometimes not
- spark sql select double column but says FloatWritable cannot cast to DoubleWritable
- Abstract static methods in Scala
- Rewrite string modifications more functional
- spark scala on windows machine
- Can HTTPS listening be enabled in Play Framework during tests?
- difference between rdd.collect().toMap to rdd.collectAsMap()?
- Why isn't .withSession in scope but it works fine when a trait?
- Possible to add udf wrapper to this dataframe operation
- what schema evolution should I use while using quill-async-mysql and play 2.6?
- Stream or Iterator to generate all strings that match a regular expression?
- Error when shading json4s-native in simple Scala project: "Symbol 'term org.json4s' is missing from the classpath."
- specs and specs2: how to implement doBefore{} in specs2?
- shapeless filter a list of options
- Choosing the last element of a list
- can't find method result on TableQuery with slick 3.0.0-RC1