score:1
Accepted answer
use p._1 instead of p(0).
val rdd = sc.parallelize(List("dog", "tiger", "lion", "cat", "spider", "eagle"), 1)
val kvRdd: RDD[(Int, String)] = rdd.keyBy(_.length)
val filterRdd: RDD[(Int, String)] = kvRdd.filter(p => p._1 == 4)
//display rdd
println(filterRdd.collect().toList)
List((4,lion))
score:1
There's a lookup method applicable to RDDs of Key-Value pairs (RDDs of type RDD[(K,V)]) that directly offers this functionality.
b.lookup(4)
// res4: Seq[String] = WrappedArray(lion)
b.lookup(5)
// res6: Seq[String] = WrappedArray(tiger, eagle)
Source: stackoverflow.com
Related Query
- Accessing a specific element of an Array RDD in apache-spark scala
- Convert RDD of Vector in LabeledPoint using Scala - MLLib in Apache Spark
- Writing JSON array of strings with a blob element in Spark Scala
- update element of array inside dataset based on condition in spark scala
- Need help to group by then sort by value on an rdd at apache spark via scala
- i want to store each rdd into database in twitter streaming using apache spark but got error of task not serialize in scala
- How to get data out of Wrapped Array in Apache Spark / Scala
- Apache Spark in Scala not printing rdd values
- Scala select n-th element of the n-th element of an array / RDD
- Flatten nested json with array into single line dataframe in Apache Spark Scala
- Apache Spark nested iterations in Scala to generate stats RDD
- How to update a specific element in Immutable Array using Scala
- how to insert element to rdd array in spark
- How to find the mean of values in an array in Scala - Apache Spark
- Convert spark scala dataset to specific RDD format
- In a Scala notebook on Apache Spark Databricks how do you correctly cast an array to type decimal(30,0)?
- Spark Scala - drop the first element from the array in dataframe
- Apache Spark (Scala): How do I grab a single element and sub-elements from a JSON RDD and store it in a new RDD?
- Extract a specific JSON structure from a json string in a Spark Rdd - Scala
- Is it possible to convert apache ignite rdd to spark rdd in scala
- Apache Spark Scala : groupbykey maintains order of values in input RDD or not
- Regex RDD using Apache Spark Scala
- Split RDD with Apache Spark Scala
- Classify every instance of a RDD | Apache Spark Scala
- Retrieve the struct from array of structs when struct field of struct type matches with specific value in spark scala
- How to Process Nested struct with Array element in Spark RDD
- IN Aggregation, based on SUM, select specific row-values in Apache Spark Dataframe, using Scala
- How to retrieve specific values from an array of string in RDD spark
- how to get all element from array of arrays of spark dataframe column scala
- How to select several element from an RDD file line using Spark in Scala
More Query from same tag
- Unable to iterate over scala list serialized with kryo
- How to get this simple regex-parser to catch a Boolean expression?
- How can Scala receive multiple parameters in a method definition?
- How to use fully formed SQL with spark structured streaming
- Log warnings in pattern match without returning Any
- How to define an empty query
- Scala Slick: Cusom mapper type DateTime to datetime
- Fast Mass File Copying in Scala with Akka
- Apache Spark/Scala Bulk inset/update to HBASE
- How to use the RangePartitioner in Spark
- Pivoting a Dataframe column transforming on a User ID Spark
- Scala: Redis client implementation with Akka futures
- FlywayDB: Migration fails with "Error executing statement", but manually running migration SQL code works
- Scala: Process dataframe while value in column meets condition
- Resolving Implicit for `Show` Typeclass Instance
- How to reduce the verbosity of Spark's runtime output?
- How can i remove duplicate tuples with scala? Cartesian Scala Spark
- Shouldn't an implicit parameter in scope be available as an implicit argument for macro-generated code?
- Migrating to 2.6: lang is not being implicitly propagated out of the request
- Index nested object with DocumentMap
- Use Java Libraries in Scala?
- create data frame with key value pair in scala
- how to get the mock TransportClient in ElasticSearch
- Scala - Error type mismatch found : List[String] required: scala.collection.IterableOnce[Nothing]
- override case class method in scala
- How to filter a dataframe in Spark scala with relational operators as variables?
- scala specs don't exit when testing actors
- Message from `ClusterReceptionist` doesn't reach `DistributedPubSubMediator`
- How to read resources in test code with Play Framework?
- N-Tuple of Options to Option of N-Tuple
