score:2
update
i have shamelessly borrowed from the answer by truhland to give this improved version of my answer that does not fail with empty or single-element lists:
def createtuple(words: list[string]): map[tuple2[string, string], int] =
words
.zip(words.drop(1))
.groupby(identity)
.mapvalues(_.length)
original
you appear to be counting adjacent pairs of words is a list of words. if so, something like this should work:
def createtuple(words: list[string]): map[tuple2[string, string], int] =
words
.sliding(2)
.map(l => (l(0), l(1)))
.tolist
.groupby(identity)
.mapvalues(_.length)
this works as follows
sliding(2)
creates a list of adjacent pairs of wordsmap
turns each pair from alist
into a tuplegroupby
groups the tuples with the same valuemapvalues
counts the number of pairs with the same value for each pair
this may not be quite what you want, but hopefully it gives an idea of how it might be done.
as a general rule, don't iterate through a list using an index, but try to transform the list into something where you can iterate through the values.
try to not create map
s element-by-element. use groupby
or tomap
.
score:1
your big problem is that words
is a list
, and yet you are indexing into it with words(i)
. that's slow. change it to be a vector
or rework your algorithm to not use indexing.
also, paircounts.exists
is slow, you should use contains
whenever possible, as it is constant time on a map.
score:1
if we first reduce your code to essence:
def createtuple(words: list[string]): map[(string, string), int] = {
val paircounts = collection.mutable.map[(string, string), int]()
for (i <- 0 until words.length - 1) {
val pair = (words(i), words(i + 1))
paircounts += (pair -> (paircounts.getorelse(pair, 0) + 1))
}
paircounts.tomap
}
to improve speed, don't use indexing on list (as mentioned elsewhere):
def createtuple(words: list[string]): map[(string, string), int] = {
val map = collection.mutable.map[(string, string), int]()
words
.zip(words.tail)
.foreach{ pair =>
map += (pair -> (map.getorelse((pair, 0) + 1)) }
map.tomap
}
Source: stackoverflow.com
Related Query
- Speeding up a method that iterates through text and creates a Map[Tuple2[String, String], Int] Scala
- Spark / Scala how to write a complex query that iterates through a dataframe and adds a column
- protected method that takes an abstract super class instance and the "access to protected method not permitted" error
- Invoking a Future inside a receive method and stopping the actor after that
- How to assert that mocked method is never called using ScalaTest and ScalaMock?
- Simplest method for text lemmatization in Scala and Spark
- How to override toString method in Scala and replace it with a dynamic text
- Method returns NodeBuffer instead of Elem and that violates the type checking rule
- Scala and Hive: best way to write a generic method that works with all types of Writable
- Method calling a function that takes a String and returns Unit
- An akka streams function that creates a sink and a source that emits whatever the sink receives
- A method that accepts both mutable and immutable map in Scala
- Spark - create list of words from text file and the word that comes immediately after it
- Can I use a ES client that creates a local node and joins a cluster over TCP on Heroku?
- spark and mill - create an additional task that creates a filtered assembly
- Scala method that takes a function as a parameter and then executes it
- Canonical way to define and execute a method that outputs Unit?
- A method in Scala that takes no arguments and uses a generic type
- How write code that creates a Dataset with columns that have the elements of an array column as values and their names being positions?
- Is it possible to call a method that is both final and protected from a unit test?
- What is and why is the fault that creates the Illegal cyclic reference error in Scala?
- Spark transformation that withdraws objects from a ListBuffer in a RDD[ListBuffer] and creates new entries in the RDD
- How to split a line of text and append that line to each element in Scala/Spark
- finding a substring in a text column that start and end with a specific string
- Difference between method and function in Scala
- Difference between using App trait and main method in scala
- How to create a Source that can receive elements later via a method call?
- Fold and foldLeft method difference
- Scala: How do I dynamically instantiate an object and invoke a method using reflection?
- Does Scala have a library method to build Option-s that takes into account empty strings?
More Query from same tag
- 22 Column limit for procedures
- Scala variable scope and for loops
- Convert List[Map[String,Map[String,Int]]] to Map[Int,Int] in Scala
- get size of an object/case class in scala
- Recursively transform elements in shapeless records based on type
- Using object SomeController extends Controller in Play framework 2.5 instead
- Why akka actor is so different from thread
- Drop into interpreter during arbitrary scala code location
- Scala is unable to type check when higher kinded types are used
- What is the difference between Exception and Throwable classes?
- sbt-assembly exclude dependsOn dependencies
- Intellij IDEA Java/Scala project subpackage display
- Keys into columns and values into rows in a Map
- Is there a way to filter a field not containing something in a spark dataframe using scala?
- How to return a function from a method by naming it when there is more than one exists?
- Heterogeneous Map in Scala, is it possible?
- Disable deprecation warnings in the Scala REPL
- Future[List[Error \/ Double]] to Future[[List[Error] \/ List[Double]] in Scala
- Akka http: how to make complete function see my Encoders?
- Does Slick Direct Embedding with Play 2.0 Work?
- Scala function definition vs application
- playframework2 and Scala - how to launch tests from IDEA?
- Ambiguous implicit in Play
- Best practice for null-checking in Scala
- can a scala self type enforce a case class type
- Scala: Compare elements at same position in two arrays
- What's the difference between a set and a mapping to boolean?
- Play Framework begginner. Trying to implement model
- Scala typing with implicits gives typing error
- Restrict Scala type to be one of a set of types