Akka unit testing strategies without mocks

Question

MAIN IDEA: How can we unit test (or re-factor to facilitate unit testing) Akka actors with fairly complex business logic?

I have been using Akka for a project at my company (some very basic stuff is in production) and have been continuously re-factoring my actors, researching, and experimenting with Akka testkit to see if I can get it right...

Basically, most of the reading I've done casually says "Man, all you need is the testkit. If you're using mocks you're doing it wrong!!" however the docs and examples are so light that I find a number of questions that are not covered (basically their examples are pretty contrived classes that have 1 method & interacts with no other actors, or only in trivial ways like input output at end of method). As an aside, if anyone can point me to a test suite for an akka app with any reasonable amount of complexity, I'd really appreciate it.

Here I will now at least attempt to detail some specific cases & would like to know what one would call the "Akka-certified" approach (but please nothing vague... I'm looking for the Roland Kuhn style methodology, if he were ever to actually dive in-depth to specific issues). I will accept strategies that involve refactoring, but please note my anxieties towards this mentioned in the scenarios.

Scenario 1 : Lateral methods (method calling another in the same actor)

case class GetProductById(id : Int)
case class GetActiveProductsByIds(ids : List[Int])

class ProductActor(val partsActor : ActorRef, inventoryActor : ActorRef) extends Actor {
  override def receive: Receive = {
    case GetProductById(pId) => pipe(getProductById(pId)) to sender
    case GetActiveProductsByIds(pIds) => pipe(getActiveProductsByIds(pIds)) to sender
  }

  def getProductById(id : Int) : Future[Product] = {
    for {
      // Using pseudo-code here
      parts <- (partsActor ? GetPartsForProduct(id)).mapTo[List[Part]]
      instock <- (inventoryActor ? StockStatusRequest(id)).mapTo[Boolean]
      product <- Product(parts, instock)
    } yield product
  }

  def getActiveProductsByIds(ids : List[Int]) : Future[List[Product]] = {
    for {
      activeProductIds <- (inventoryActor ? FilterActiveProducts(ids)).mapTo[List[Int]]
      activeProducts <- Future.sequence(activeProductIds map getProductById)
    } yield activeProducts
  }
}

So, basically here we have what are essentially 2 fetch method, one singular and one for multiple. In the singular case, testing is simple. We set up a TestActorRef, inject some probes in the constructor and just make sure that the right message chain is firing.

My anxiety here comes from the multiple fetch method. It involves a filtering step (to fetch only active product IDs). Now, in order to test this, I can set up the same scenario (TestActorRef of ProductActor with probes replacing the actors called for in the constructor). However, to test the message passing flow I have to mock all of the message chaining for not only the response to FilterActiveProducts but all of the ones that have already been covered by the previous test of the "getProductById" method (not really unit testing then, is it?). Clearly this can spiral out of control in terms of the amount of message mocking necessary, and it would be far easier to verify (through mocks?) that this method simply gets called for every ID that survives the filter.

Now, I understand this can be solved by extracting out another actor (create a ProductCollectorActor that gets for multiple IDs, and simply calls down to the ProductActor with a single message request for each ID that passes the filter). However, I've calculated this & if I were to do extractions like this for every hard-to-test set of sibling methods I have I will end up with dozens of actors for relatively small amount of domain objects. The amount of boilerplate overhead would be a lot, plus the system will be considerably more complex (many more actors just perform what are essentially some method compositions).

Aside : Inline (static) logic

One way I've tried to settle this is by moving inline (basically anything that is more than a very simple control flow) into the companion or another singleton object. For example, if there was a method in the above method that was to filter out products unless they matched a certain type, I might do something like the following:

object ProductActor {
  def passthroughToysOnly(products : List[Product]) : List[Toy] =
    products flatMap {p => 
      p.category match {
        case "toy" => Some(p)
        case _ => None
      }
    }
}

This can be unit tested in isolation pretty well, and can actually allow for testing of pretty complex units so long as they don't call out to other actors. I'm not a huge fan of putting these far from the logic that uses them (should I put it in the actual actor & then test by calling underlyingActor?).

Overall, it still leads to the problem that in doing more naive message-passing based tests in the methods that actually call this, I essentially have to edit all my message expectations to reflect how the data will be transformed by these 'static' (I know they're not technically static in Scala but bear with me) methods. This I guess I can live with since it's a realistic part of unit testing (in a method that calls several others, we are probably checking the gestalt combinatorial logic in the presence of test data with varying properties).

Where this all really breaks down for me is here --

Scenario 2 : Recursive algorithms

case class GetTypeSpecificProductById(id : Int)

class TypeSpecificProductActor(productActor : ActorRef, bundleActor : ActorRef) extends Actor {
  override def receive: Receive = {
    case GetTypeSpecificProductById(pId) => pipe(getTypeSpecificProductById(pId)) to sender
  }

  def getTypeSpecificProductById(id : Int) : Future[Product] = {
    (productActor ? GetProductById(id)).mapTo[Product] flatMap (p => p.category match {
        case "toy" => Toy(p.id, p.name, p.color)
        case "bundle" => 
          Bundle(p.id, p.name, 
            getProductsInBundle((bundleActor ? GetProductIdsForBundle(p.id).mapTo[List[Int]))
      }
    )
  }

  def getProductsInBundle(ids : List[Int]) : List[Product] =
    ids map getProductById
}

So yes there is a bit of pseudocode here but the gist is that now we have a recursive method (getProductId is calling out to getProductsById in the case of a bundle, which is again calling getProductId). With mocking, there are points where we could cut off recursion to make things more testable. But even that is complex due to the fact that there are actor calls within certain pattern-matches in the method.

This is really the perfect storm for me.... extracting the match for the "bundle" case into a lower actor may be promising, but then also means that we now need to deal with circular dependency (bundleAssembly actor needs typeSpecificActor which needs bundleAssembly...).

This may be testable via pure message mocking (creating stub messages where I can gauge what level of recursion they will have & carefully designing this message sequence) but it will be pretty complex & worse if more logic is required than a single extra actor call for the bundle type.

Remarks

Thanks ahead of time for any help! I am actually very passionate about minimal, testable, well-designed code and am afraid that if I try to achieve everything via extraction I will still have circular issues, still not be able to really test any inline/combinatorial logic & my code will be 1000x more verbose than it could have been with lots of boilerplate for tiny single-to-the-extreme-responsibility actors. Essentially, the code will all be written around test structure.

I am also very cautious about over-engineered tests, because if they are testing intricate message sequences rather than method calls (which I'm not sure how to expect simple semantic calls except with mocks) the tests may succeed but won't really be a true unit tests of core method functionality. Instead it will just be a direct reflection of control flow constructs in the code (or the message passing system).

So maybe it is that I am asking too much of unit testing, but please if you have some wisdom set me straight!

Answer 1

I disagree with your statement "I'm not a huge fan of putting these far from the logic that uses them".

I find this to be an essential component of unit testing and code organization.

Jamie Allen, in Effective Akka, states the following about externalizing business logic (emphasis mine):

This has a few added benefits. First of all, not only can we we write useful unit tests, but we can also get meaningful stack traces that say the name where the failure occurred in the function. It also prevents us from closing over external state, since everything must be passed as an operand to it. Plus, we can build libraries of reusable functions that reduce code duplication.

When writing code I take it a step further than your example and move the business logic into a separate package:

package businessLogic

object ProductGetter {
  def passthroughToysOnly(products : List[Product]) : List[Toy] =
    products flatMap {p => 
      p.category match {
        case "toy" => Some(p)
        case _ => None
    }
  }
}

This allows for changing the concurrency methodology to Futures, Java threads, or even some not yet created concurrency library without having to refactor my business logic. The business logic packages become the "what" of the code, the akka libraries become the "how".

If you isolate the business logic then all receive methods become simple "routers" of messages to external functions. Therefore, if you hammer at your business logic with unit tests the only testing you need to do of your Actors is to ensure the case patterns are matching correctly.

Addressing your specific problem: I would remove the getActiveProductsByIds from the Actor. If the user of the Actor wants to only get active products leave it to them to filter the ids first. Your Actor should do 1 thing only: GetProductById. Quoting Allen again:

It is very easy to make an actor perform addition tasks - we simply add new messages to its receive block to allow it to perform more and different kinds of work. However, doing so limits your ability to compose systems of actors and define contextual grouping. Keep your actors focused on a single kind of work, and in doing so, allow yourself to use them flexibly.

Answer 2

First of all, this is a very interesting question. Akka documentation is very good in general, and the testing part has many insightful notes along the way to avoid common pitfalls and suggesting best practices.

The other day I was reading about this, and found a suggestion that I didn't try before: using the observer pattern. The idea is to leave your Actors to only care about messaging (which you don't need to test, Akka team will do it for you ;) and broadcast events to subscribers. This way your logic gets completely isolated from Actors, hence making it much easier to test.

Note: I haven't tried this in a production system, but since you mentioned that only very basic stuff is in your production system this might be worth a shot.