Ember.js Acceptance Testing not waiting for asynchronous data operations

Question

When using the Emberfire (Firebase) adapter, I'm getting various errors that indicate the tests are not waiting for data operations to complete.

eg: Error: Assertion Failed: You can only unload a record which is not inFlight. when I try to create, check, and then delete a record

Also:

FIREBASE WARNING: Exception was thrown by user callback. Error: Called stop() outside of a test context at Object.extend.stop (http://localhost:4200/assets/test-support.js:3000:10) at exports.default._emberTestingAdaptersAdapter.default.extend.asyncStart

These errors do not occur while manually navigating through my app, nor when using the standard ember data adapter.

What is causing these errors and how do I avoid them?

Edit: although the symptoms are a bit different (no error thrown), it sounds like this problem may have the same root cause as the errors I've been seeing.

Answer 1

tl;dr

To work around the issue, I've been using a custom test waiter. You can install it with ember install ember-cli-test-model-waiter (works with Ember v2.0+)

Longer answer:

The root cause of these problems is that the ember testing system doesn't know how to handle Firebase's asynchronicity. With most adapters, this isn't a problem, because the testing system instruments AJAX calls and ensures they have completed before proceeding, but this doesn't work with Firebase's websockets communication.

So although these errors don't occur when interacting manually, I believe they technically could if it were possible to click fast enough.

These problems are known to occur with ember-pouch and will likely also occur with other non-AJAX adapters (eg. localstorage adapters (1, 2), or any other websockets-based adapter. It may occur with the fixture adapter, but that may return results immediately and so not trigger this problem). It also happens with other asynchronous processes, like liquid-fire animations (fixed in a similar way)

The custom test waiter I mentioned in the tl;dr works by waiting for all models to resolve before proceeding with the test, and so should work with all of these non-AJAX adapters.

For more background on how ember's testing deals with asynchronicity under the hood, Cory Forsyth has a helpful blog post, and this gist gives another more flexible solution approach but that requires more manual book-keeping.