EmberJS test fails first time running in phantomjs

2
votes

Problem

I have a /login route that uses ember-simple-auth to implement authentication. During testing ember-cli-mirage is used to mock the backend. The user logs in by providing their email address and password.

In total I have 4 acceptance tests for this route, similar to the test below:

test('should show error message for invalid email', function(assert) {
  visit('/login');
  fillIn('input#email', 'invalid-email');
  fillIn('input#password', 'invalid-password');
  click('button.button');
  andThen(function() {
    assert.equal(find('div.notification').text(), "Invalid email/password");
  });
});

When I run the tests using ember t only the first test in the file fails. If I comment this test out, the next one fails, and so on. If I run the tests in server mode with ember t -s the same test fails; however, when I press enter to re-run the tests, all the tests pass.

The failure message is always the same, shown below:

not ok 7 PhantomJS 2.1 - Acceptance | login: should show error message for invalid email
    ---
        actual: >

        expected: >
            Invalid email/password
        stack: >
            http://localhost:7357/assets/tests.js:22:19
            andThen@http://localhost:7357/assets/vendor.js:48231:41
            http://localhost:7357/assets/vendor.js:48174:24
            isolate@http://localhost:7357/assets/vendor.js:49302:30
            http://localhost:7357/assets/vendor.js:49258:23
            tryCatch@http://localhost:7357/assets/vendor.js:68726:20
            invokeCallback@http://localhost:7357/assets/vendor.js:68738:21
            publish@http://localhost:7357/assets/vendor.js:68709:21
            http://localhost:7357/assets/vendor.js:48192:24
            invoke@http://localhost:7357/assets/vendor.js:10892:18
            flush@http://localhost:7357/assets/vendor.js:10960:15
            flush@http://localhost:7357/assets/vendor.js:11084:20
            end@http://localhost:7357/assets/vendor.js:11154:28
            run@http://localhost:7357/assets/vendor.js:11277:19
            run@http://localhost:7357/assets/vendor.js:32073:32
            http://localhost:7357/assets/vendor.js:48783:24
        Log: |

After all the tests have run, test emits an exception:

# tests 60
# pass  59
# skip  0
# fail  1
Not all tests passed.
Error: Not all tests passed.
    at EventEmitter.getExitCode (/home/jon/projects/jonblack/wishlist-web/node_modules/testem/lib/app.js:434:15)
    at EventEmitter.exit (/home/jon/projects/jonblack/wishlist-web/node_modules/testem/lib/app.js:189:23)
    at /home/jon/projects/jonblack/wishlist-web/node_modules/testem/lib/app.js:103:14
    at tryCatcher (/home/jon/projects/jonblack/wishlist-web/node_modules/testem/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/home/jon/projects/jonblack/wishlist-web/node_modules/testem/node_modules/bluebird/js/release/promise.js:510:31)
    at Promise._settlePromise (/home/jon/projects/jonblack/wishlist-web/node_modules/testem/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (/home/jon/projects/jonblack/wishlist-web/node_modules/testem/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (/home/jon/projects/jonblack/wishlist-web/node_modules/testem/node_modules/bluebird/js/release/promise.js:691:18)
    at Async._drainQueue (/home/jon/projects/jonblack/wishlist-web/node_modules/testem/node_modules/bluebird/js/release/async.js:138:16)
    at Async._drainQueues (/home/jon/projects/jonblack/wishlist-web/node_modules/testem/node_modules/bluebird/js/release/async.js:148:10)
    at Immediate.Async.drainQueues (/home/jon/projects/jonblack/wishlist-web/node_modules/testem/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:637:20)
    at tryOnImmediate (timers.js:610:5)
    at processImmediate [as _immediateCallback] (timers.js:582:5)

It seems odd that this is emitted for tests failing rather than just reporting the test failure, so perhaps it's related.

Running the tests in Firefox and Chromium work, as does running the application in development mode and logging in manually. The problem is limited to phantomjs.

I have other acceptance tests for another route and these all pass. It seems limited to the /login route, suggesting that it is possibly related to authentication.

Debugging

I've tried debugging by adding pauseTest() to the test and "phantomjs_debug_port": 9000 to testem.js but both Firefox and Chromium do nothing when I use the debug console. This might be my lack of experience debugging phantomjs, but I would at least expect it to give me an error - it literally does nothing.

It feels as though there is a timing issue between phantomjs and something, possible ember-simple-auth, in my Ember app.

I'm not that experienced debugging phantomjs problems nor Ember acceptance test failures, so any help is appreciated.

Versions

ember-cli 2.10.0
ember-simple-auth 1.1.0
ember-cli-mirage 0.2.4

Update 1

The button is inside a login-form component:

<form {{action 'login' on='submit'}}>
  <p class="control has-icon">
    {{input value=email id='email' placeholder='email' class='input'}}
    <i class="fa fa-envelope"></i>
  </p>
  <p class="control has-icon">
    {{input value=password id='password' placeholder='password'
            type='password' class='input'}}
    <i class="fa fa-lock"></i>
  </p>
  <p class="control">
  <button class="button is-success" disabled={{isDisabled}}>Log In</button>
  </p>
</form>

The component's login action just calls the passed in login handler:

import Ember from 'ember';

export default Ember.Component.extend({
  email: "",
  password: "",

  isDisabled: Ember.computed('email', 'password', function() {
    return this.get('email') === "" || this.get('password') === "";
  }),

  actions: {
    login() {
      var email = this.get('email');
      var password = this.get('password');
      this.attrs.login(email, password);
    }
  }
});

Which is the authenticate method in the login controller:

import Ember from 'ember';

export default Ember.Controller.extend({
  session: Ember.inject.service(),

  actions: {
    authenticate(email, password) {
      this.get('session').authenticate('authenticator:oauth2', email, password).catch((data) => {
        this.set('errors', data['errors']);
      });
    }
  }
});

Update 2

As suggested by Daniel I added a delay to the test:

test('should show error message for invalid email', function(assert) {
  visit('/login');
  fillIn('input#email', 'invalid-email');
  fillIn('input#password', 'invalid-password');
  click('button.button');
  andThen(function() {
    Ember.run.later(this, function() {
      assert.equal(find('div.notification').text(), "Invalid email/password");
    }, 0);
  });
});

Using only Ember.run.later the test still failed, but putting that inside the andThen causes it to pass. Have you noticed the bizarre part? The delay is 0 milliseconds.

I still want to find an explanation for this because I don't trust that this will run the same on whatever machine the tests run on.

Update 3

Today I had a surprise: suddenly the tests were working again!

I added a new route with acceptance tests. The route itself is an authenticated route, so the tests use the authenticateSession test helper from ember-simple-auth to authenticate.

when I remove the tests that use this helper, the error returns!.

I'm not sure what this means. It feels like the issue is with ember-simple-auth, but it might also be a giant coincidence that the helper resolves another timing issue.

Down the rabbit hole we go...

Update 4

Below is the configuration for the auth endpoints in ember-cli-mirage:

this.post('/token', function({db}, request) {
  var data = parsePostData(request.requestBody);

  if (data.grant_type === 'password') {
    // Lookup user in the mirage db
    var users = db.users.where({ email: data.username });
    if (users.length !== 1) {
      return new Mirage.Response(400, {'Content-Type': 'application/json'}, {
        errors: [{
          id: 'invalid_login',
          status: '400',
          title: 'Invalid email/password',
        }]
      });
    }
    var user = users[0];

    // Check password
    if (data.password === user.password) {
      if (!user.active) {
        return new Mirage.Response(400, {'Content-Type': 'application/json'}, {
          errors: [{
            id: 'inactive_user',
            status: '400',
            title: 'Inactive user',
          }]
        });
      } else {
        return new Mirage.Response(200, {
          'Content-Type': 'application/json'
        }, {
          access_token: 'secret token!',
          user_id: user.id
        });
      }
    } else {
      return new Mirage.Response(400, {'Content-Type': 'application/json'}, {
        errors: [{
          id: 'invalid_login',
          status: '400',
          title: 'Invalid email/password',
        }]
      });
    }
  } else {
    return new Mirage.Response(400, {'Content-Type': 'application/json'}, {
      errors: [{
        id: 'invalid_grant_type',
        status: '400',
        title: 'Invalid grant type',
      }]
    });
  }
});

this.post('/revoke', function(db, request) {
  var data = parsePostData(request.requestBody);

  if (data.token_type_hint === 'access_token' ||
      data.token_type_hint === 'refresh_token') {
    return new Mirage.Response(200, {'Content-Type': 'application/json'});
  } else {
    return new Mirage.Response(400, {'Content-Type': 'application/json'},
                               {error: 'unsupported_token_type'});
  }
});

Update 5

Here's my config/environment.js file:

/* jshint node: true */

module.exports = function(environment) {
  var ENV = {
    modulePrefix: 'wishlist-web',
    environment: environment,
    rootURL: '/',
    locationType: 'auto',
    EmberENV: {
      FEATURES: {
      },
      EXTEND_PROTOTYPES: {
        // Prevent Ember Data from overriding Date.parse.
        Date: false
      }
    },

    APP: {
    }
  };

  if (environment === 'development') {
  }

  if (environment === 'test') {
    // Testem prefers this...
    ENV.locationType = 'none';

    // keep test console output quieter
    ENV.APP.LOG_ACTIVE_GENERATION = false;
    ENV.APP.LOG_VIEW_LOOKUPS = false;

    ENV.APP.rootElement = '#ember-testing';
  }

  if (environment === 'production') {
    ENV.ServerTokenEndpoint = 'http://localhost:9292/token';
    ENV.ServerTokenRevocationEndpoint = 'http://localhost:9292/revoke';
    ENV.ApiHost = 'http://localhost:9292';
  }

  return ENV;
};
2
ember.jsphantomjsember-simple-auth
When you clicked the button.button, how do you create a promise? Can you provide the code that creates the promise? (My guess is the promise cannot be resolved in phantomjs environment.) – ykaragol

2 Answers

1
votes

You have few things to try here to debug this issue.

  1. You could remove {{isDisabled}} from button to make sure it's not disabled when you try to click it.

  2. Use setTimeout instead of andThen and see if it's timing issue.

  3. Replace authenticate action code with nothing, to make sure it isn't causing your test to fail.

You could also rewrite test to put your assert.ok after some event in JavaScript. For example you could mock authenticate action or observer errors property. You can do this by using lookups or registers in acceptance environment - tests from one of my Ember CLI addons could help you - ember-link-action/tests/acceptance/link-action-test.js.

Edit

Having seen what worked for you experience tells me that you should try 2 things.

For this code:

andThen(function() {
    Ember.run.later(this, function() {
      assert.equal(find('div.notification').text(), "Invalid email/password");
    }, 0);
  });

You could try Ember.run.scheduleOnce('afterRender', this, () => { ... assert here } instead of using Ember.run.later. Or you could try using just Ember.run instead of Ember.run.later.

Conclusion: The key to fixing this issue could be putting your assertion in Ember Run Loop.

1
votes

I would assume that the error you're seeing (Invalid email/password) is the (mock) server response and indicates something is wrong with either the mock or the credentials you're using in the test.

I'd also not use mirage for mocking the authentication request. mirage (just like Jason API) is resource based and not something that's well suited for authentication.