Get rid of Scala Future nesting

Question

Again and again I am struggling when a function relies on some future results. This usually boils down to a result like Future[Seq[Future[MyObject]]]

To get rid of that I now use Await inside a helper function to get a non-future object out and reduce the nesting.

It looks like this

def findAll(page: Int, perPage: Int): Future[Seq[Idea]] = {
    val ideas: Future[Seq[Idea]] = collection.find(Json.obj())
    // [...]

    ideas.map(_.map { // UGLY?
      idea => {
        // THIS RETURNED A Future[JsObject] before
        val shortInfo: JsObject = UserDao.getShortInfo(idea.user_id)
        idea.copy(user_data = Some(shortInfo))
      }
    })
}

This code works but to me it looks quite hacky. The two map calls are another flaw. I spent hours trying to figure out how to keep this completely asynchronous and returning a simple future Seq. How can this be solved using Play2 best practices?

Edit To make the usecase more clear:

I have an object A from mongodb (reactivemongo) and want to add information coming from another call to mongodb getShortInfo. It's a classical "get user for this post" case that would be solved with a join in RDBMS. getShortInfo naturally would produce a Future because of the call to the db. To reduce the nesting within findAll I used Await(). Is this a good idea?

findAll is called from an asynchronous Play action, converted into Json and sent over the wire.

def getIdeas(page: Int, perPage: Int) = Action.async {

  for {
    count <- IdeaDao.count
    ideas <- IdeaDao.findAll(page, perPage)
  } yield {
    Ok(Json.toJson(ideas))
  }
}

So I think returning a Seq[Future[X]] from findAll won't bring better performance as I have to wait for the result anyways. Is this correct?

The usecase in short: Take a Future call returning a Sequence, use each element of the result to create another Future call, return the result to an asynchronous action in a way that no blocking situations should occur.

Sorry, I'm not clear on what the question is. If you just want to filter or process the values of the wrapped Seq in the future, then your solution seems fine. There's nothing wrong with having two map calls; that is normal and necessary, since the first map is over the Future, and the second is over the Seq. I don't see any need for Await here; are you saying that is already happening inside UserDao.getShortInfo? — David Soergel
So I think your question is: if you had not used Await inside UserDao.getShortInfo, then it would have produced Future[JsObject]. and the output of this FindAll would have been Future[Seq[Future[Idea]]], which is not what you want. Then the question arises what you do want. Are you sure you need Future[Seq[Idea]], which can only resolve once all of the elements have been processed? Perhaps you'd prefer Seq[Future[Idea]], since the elements appear to be processed independently anyway? — David Soergel
Meanwhile I found out that one can always get rid of all blocking operations. With some of the techniques inside this answers and flatmapping over operations that return a future you never will need to use await again. — DanielKhan
Note: scala 2.12 will have a Future.flatten method: stackoverflow.com/a/36079431/6309 — VonC

stew stew · Accepted Answer · 2013-11-29T03:17:54

Two handy functions on the Future companion object you should know could help here, the first, and easier to wrap your head around is Future.sequence. It takes a sequnce of futures and returns a Future of a sequence. If are ending up with a Future[Seq[Future[MyObject]]], lets call that result. then you can change this to a Future[Future[Seq[MyObject]]] with result.map(Future.sequence(_))

Then to collapse a Future[Future[X]] for any X, you can run "result.flatMap(identity)", in fact, you can do this for any M[M[X]] to create a M[X] as long as M has flatMap.

Another useful function here is Future.traverse. It is basically the result of taking a Seq[A], mapping it to a Seq[Future[B]], then running Future.sequence to get a Future[Seq[B]] So in your example, you'd have:

ideas.map{ Future.traverse(_){ idea =>
    /*something that returns a Future[JsObject]*/
} }.flatMap(identity)

However, many times when you are running flatMap(identity), you could be turning a map into a flatMap, and this is the case here:

ideas.flatMap{ Future.traverse(_) { idea =>
    /*something that returns a Future[JsOjbect]*/
} }

Get rid of Scala Future nesting

2 Answers