score:2
when using database.forconfig a different value for the queue size can be provided.
score:9
1000 queued tasks seems like a lot to me. clearly, slick is using an executor with a fixed size queue (1000 elements), and you're running into that limit because tasks are not being retired quickly enough.
the most obvious cause is sql execution times. if you can get your sql execution times down, you'll buy yourself a lot of headroom in the queue.
typically, this would be done database-side by checking query execution plans, and depending on the database, asking the database what the long running queries are.
on the hikaricp side, you might want to enable dropwizard metrics (not sure how to do that through slick, tho) and enable a dropwizard log reporter to log pool statistics every 10 seconds or so.
probably the most interesting metric there would be usage, as that will show you how long connections are out of the pool between getconnection() and close(). as you tune your database and/or queries, you want to see that number start dropping.
one crucial point is, if the database cannot keep up with your application load, increasing the slick queue from 1000 to 5000 (or even 10000) will buy you nothing except a small amount of time before it reaches that limit. you have to find the source of the performance bottleneck and eliminate it, such that queues are retired faster than your application generates them (excepting temporal spikes of course).
Source: stackoverflow.com
Related Query
- What to do when hitting the queue size of slick?
- Apache Spark What happens when repartition($"key") is called when size of all records per key is greater than the size of a single partition?
- What is the formal difference in Scala between braces and parentheses, and when should they be used?
- What are the relationships between Any, AnyVal, AnyRef, Object and how do they map when used in Java code?
- What are the precise rules for when you can omit parenthesis, dots, braces, = (functions), etc.?
- Scala - What is the difference between size and length of a Seq?
- What does the @elidable annotation do in Scala, and when should I use it?
- What is the difference between inSet and inSetBind in Slick
- What really happens behind the Scala runtime/REPL when running a '.scala' program?
- What do the % and %% operators do when setting up SBT dependencies?
- What does the keyword 'implicit' mean when it's placed in front of lambda expression parameter?
- what is the difference between HashSet and Set and when should each one be used?
- What is the difference between "class C extends A with B" and "class C extends B" when trait B extends trait A
- What is the appropriate return value when the pattern matched is Nil and we want to return Nil?
- Play Framework: What happens when requests exceeds the available threads
- What happens to a variable assignment when I call a method that returns an instance of a case class holding the said variable?
- What is the best way to deal with compsite keys when using Salat with MongoDB?
- What causes this Scala program to become suspended due to tty output when run in the background?
- In Scala, what happens when an inner object extends the enclosing class?
- What does it mean when macro annotation cannot be used in the same compilation that defines it?
- What is the running time for size on Scala's ListBuffer?
- What is the principle ? When Spark deal with data bigger than memory capacity?
- What concrete datatype is created when I use the Seq[T] constructor?
- What 28 frames are elided when dividing by zero in the REPL?
- Scala's collection's sliding() is inconsistent when the window size is greater than step
- What does the scaladoc mean when it says "All operations are guaranteed to be performed in a single-threaded manner"?
- What is the size of empty class in scala?
- Why does the task size keep growing when using updateStateByKey?
- What is the correct way to use typeclass syntax when you have a typeclass within a typeclass?
- Function returns the wrong value only when size of array input is 1
More Query from same tag
- Scala3 inline if reuse result of function
- What is gatling's "inferHtmlResources" resource limits?
- How can I use `Class.forName()` to get classes defined in a `ToolBox`?
- Why PrefixMap in scala examples has extends?
- Can I use Lift's FieldSerializer to change a field on serialization?
- How to use custom spark-defaults.conf settings
- Spark/Scala : String field is nullable , but a Double field is non-nullable
- Filtering One RDD based on another RDD using regex
- Check if a string is blank or doesn't exist in Scala
- How to log incoming request and response?
- Type equality in the Scala 2.10 Reflection API
- How to read files from resources folder in Scala?
- How to traverse up a single child node using scala.xml.Node
- How to create and use a multi-dimensional array in Scala?
- Fastest way to convert Scala array to Java array
- Getting AnalysisException on dataset
- How to avoid mutable local variables in Scala?
- Converting a vector column in a dataframe back into an array column
- How to take Transpose of a Dataset in scala?
- How to get counter of each word in windowed stream on Flink
- Scala Join Arrays and Reduce on Value
- Get or create child actor by ID
- when should lazy val be used rather than a function
- Extending Inner class
- How to understand "case id ~ username => _" in anorm?
- Call Java method from Scala named "clone"
- use function or class of scala in spark: org.apache.spark.SparkException: Task not serializable
- Could not find implicit in Scala Tagless Final, Implicit, unit test
- Alternate Use to 'return' in Scala?
- Analog to Ruby bundler in Scala