score:4
each entry in the rdd is a string. so comparing it to "name" will always fail, as it's "name"+some digits.
what you need is map
to iterate over the rdd and return a new value for each entry. and that new value should be the string, without the first 4 characters, and converted to long.
putting that all together, we get
name.map(_.drop(4).tolong)
if you don't know the first four characters will be "name", you might want to check that first. what you need then depends on what you want to do with rows that don't have name as the first four, but something like
name.filter(_.startswith("name")).map(_.drop(4).tolong)
score:3
the method stripprefix will remove a given prefix from a string (and do nothing if the string does not start with that prefix.
so you achieve what you need by:
val name_clean = name.map(_.stripprefix("name").tolong)
Source: stackoverflow.com
Related Query
- Scala RDD String manipulation
- Scala Spark : How to create a RDD from a list of string and convert to DataFrame
- Simple string manipulation in Scala
- Extract a specific JSON structure from a json string in a Spark Rdd - Scala
- Scala - Convert String to Date in Spark RDD
- get count of common string from two RDD in scala
- Scala string manipulation
- Spark scala filter multiple rdd based on string length
- Idiomatic way to convert an InputStream to a String in Scala
- Remove Characters from the end of a String Scala
- How to insert double quotes into String with interpolation in scala
- What is the idiomatic scala way of finding, if a given string contains a given substring?
- Better String formatting in Scala
- How to check to see if a string is a decimal number in Scala
- What's the difference between raw string interpolation and triple quotes in scala
- How can I construct and parse a JSON string in Scala / Lift
- Scala - Get last two characters from string
- How to find if a Scala String is parseable as a Double or not?
- String interpolation in Scala 2.10 - How to interpolate a String variable?
- Convert list in Scala to a formatted string
- Is string concatenation in scala as costly as it is in Java?
- Generating a class from string and instantiating it in Scala 2.10
- scala String to scala.xml.Elem
- Scala String vs java.lang.String - type inference
- Hex String to Int,Short and Long in Scala
- akka HttpResponse read body as String scala
- Computing the MD5 hash of a string in scala
- Why are List and String identifiers named "xs" (in Scala and other languages)?
- Scala safe way of converting String to Enumeration value
- Scala - case match partial string
More Query from same tag
- What is the benefit of using Futures over parallel collections in scala?
- How do I exclude package from publishing with sbt?
- play framework listen on localhost
- What's Int.MaxValue between friends?
- Parse Json Sequence without keys in playframework
- Purpose of `render` in json4s
- KAFKA STREAM: UnsatisfiedLinkError on Lib Rocks DB
- How to scrape all texts from <a href> to List with net.ruippeixotog.scalascraper
- add values from struct keys spark
- http4s: What is difference between actual and expected value?
- Explanation of "unbound placeholder parameter"
- Scala symbol to java enum
- Scala: What is the most efficient way convert a Map[K,V] to an IntMap[V]?
- need to get Array elements in Tab delimited Form
- Scala: Add values to sortedSet or array using for loop
- How I can deal with Tuple in Spark Streaming?
- How to represent an XML entity in a Scala Class beautifully?
- Enumerator vs Iterator in scala and java
- Declare variable of type String* in scala
- Using python scipy from spark scala
- Partially applying type parameters
- Game duration algorithm
- Expression returning List instead of Array
- Continuous trigger does not work with Spark Structured Streaming with valid sinks
- Accessing AWS S3 with temporary credentials
- Azure Synapse - How to stop an Apache Spark application / notebook?
- Scala - TreeSet of tuples
- SBT plugin - User defined configuration for Command via their build.sbt
- Mean Squared Error (MSE) returns a huge number
- Where did Option[T] come from in Scala?