[Solved]-How to reduce context switches when traversing a Seq of Scala futures-scala

The best solution is to change the Database access so that multiple objects are fetched within a single Future. Even if there is a limit on the number of objects that can be fetched, it is still going to drastically reduce the overheads.

A neat trick to avoid context switching when using Scala Futures consists in using parasitic as an ExecutionContext, which "steals execution time from other threads by having its Runnables run on the Thread which calls execute and then yielding back control to the caller after all its Runnables have been executed". parasitic is available since Scala 2.13 but you can easily understand it and port it to pre-2.13 projects by looking at its code (here for version 2.13.1). A naive but working implementation for pre-2.13 projects would simply run the Runnables without taking care of dispatching them on a thread, which does the trick, as in the following snippet:

object parasitic212 extends ExecutionContext {

  override def execute(runnable: Runnable): Unit =
    runnable.run()

  // reporting failures is left as an exercise for the reader
  override def reportFailure(cause: Throwable): Unit = ???

}

The parasitic implementation is of course more nuanced. For more insight into the reasoning and some caveats about its usage I would suggest you refer to the PR the introduced parasitic as a publicly available API (it was already implemented but reserved for internal use).

Quoting the original PR description:

A synchronous, trampolining, ExecutionContext has been used for a long time within the Future implementation to run controlled logic as cheaply as possible.

I believe that there is a significant number of use-cases where it makes sense, for efficiency, to execute logic synchronously in a safe(-ish) way without having users to implement the logic for that ExecutionContext themselves—it is tricky to implement to say the least.

It is important to remember that ExecutionContext should be supplied via an implicit parameter, so that the caller can decide where logic should be executed. The use of ExecutionContext.parasitic means that logic may end up running on Threads/Pools that were not designed or intended to run specified logic. For instance, you may end up running CPU-bound logic on an IO-designed pool or vice versa. So use of parasitic is only advisable when it really makes sense. There is also a real risk of hitting StackOverflowErrors for certain patterns of nested invocations where a deep call chain ends up in the parasitic executor, leading to even more stack usage in the subsequent execution. Currently the parasitic ExecutionContext will allow a nested sequence of invocations at max 16, this may be changed in the future if it is discovered to cause problems.

As suggested in the official documentation for parasitic, you're advised to only use this when the executed code quickly returns control to the caller. Here is the documentation quoted for version 2.13.1:

WARNING: Only ever execute logic which will quickly return control to the caller.

This ExecutionContext steals execution time from other threads by having its Runnables run on the Thread which calls execute and then yielding back control to the caller after all its Runnables have been executed. Nested invocations of execute will be trampolined to prevent uncontrolled stack space growth.

When using parasitic with abstractions such as Future it will in many cases be non-deterministic as to which Thread will be executing the logic, as it depends on when/if that Future is completed.

Do not call any blocking code in the Runnables submitted to this ExecutionContext as it will prevent progress by other enqueued Runnables and the calling Thread.

Symptoms of misuse of this ExecutionContext include, but are not limited to, deadlocks and severe performance problems.

Any NonFatal or InterruptedExceptions will be reported to the defaultReporter.