Client - server integration testing: mock or not?

Question

I'm working on project with two applications: android app (client) and rest service (server). My android app consumes my rest service.

Both applications are tested separately to ensure they're doing their business as expected. During server tests I prepare requests and check server responses. During client tests I set up a simple http mock server and test client's requests against different mocked responses.

Now, this technique works pretty well. It gives me a flexibility I like. I can use different test frameworks and continuous integration environments. But there is one weak point. In both (client and server) test cases I specify the same api. I assume that e.g.

GET /foo-list.json

will return HTTP 200 with json

[{
    id: 1,
    name: foo1,
}, {
    id: 2,
    name: foo2
}]

So I repeat myself. If I change a response format my client tests won't fail.

My question is about good practices in testing this kind of scenario. How to make true integration tests without sacrificing flexibility of independent tests. Should I test client with mocked server or with a real instance of my rest service?

Please share your professional experience.

Answer 1

In your scenario you should continue to write unit tests to test individual classes, and integration tests to test the inter-operation between multiple application layers (e.g. business and database layers).

You ask:

"How to make true integration tests without sacrificing flexibility of independent tests"

All of your code should should use abstractions, so that you can use dependency injection to unit test classes in complete isolation using mock dependencies. The use of mocks will ensure that these tests will remain independent i.e. not coupled to any other classes. Hence taking this approach, the integration tests, which would use your final concrete classes, would not affect the unit tests which use the mocked classes.

Also:

"Should I test client with mocked server or with a real instance of my rest service?"

In addition to unit and integration tests you should also perform client-server integration testing; I use automated acceptance testing for doing this. Using a test framework such as Cucumber (also check out calabash-android, which is written specifically to test mobile applications) you can write tests which would test specific features and scenarios which would interact with both the client (your Android application) and server (your RESTful service). These client-server integration tests would start-up and stop concrete instances of the client and server.

Answer 2

Mocks are for unit testing. Your description of the tests with the mocks describes exactly that. You test the client and server as separate units.

Integration testing tests if the units work well together. Since the interface is a REST interface, mocking makes no sense then, you have to test the real thing over HTTP.

See also What is the difference between integration and unit tests?

Answer 3

If your service is based in Java, I'd strongly recommend looking into the Spock framework, for mocking any sort of calls that might be coming from the client. Since Spock is just an extension of jUnit, you might also be able to use it for Android (though, to be fair I've never done Android development)

I'd say you want to do two things. Integration testing and Unit testing. Integration testing would attempt to bring up the android application and cause it to make service calls, ensuring the contexts interact with each other kindly.

However, in your regular commits, I'd suggest unit testing that mocks away everything but the class under test. Spock makes this pretty easy to do, and since it's built on top of jUnit all it takes a jar.

Answer 4

There is no reason you can't run automated end to end tests with a real service instance. You can run a real service instance on the same test machine you are using to run the unit tests, perhaps in the same container. You can set up configuration to use a different URL for the server instance for running automated end to end testing.

Why would you want to do the extra work of creating the mock service if you can run them against the real service?

I would only create a mock service, if the service was an external service over which I had no control!