If Protractor is replacing Angular Scenario Runner for E2E testing, does that mean I will still be able to use it with Karma as my E2E testing framework ?
105
votes
IMO you should be able to. Karma is just the test runner. Once you write/ build your test cases in Protractor.... karma should be able to just run it when asked. The same way you can ask Karma to run present E2E tests.
– skeep
@skeep Webdriver spawns and connects to browsers via WebDriver protocol - something Karma doesn't support (it uses WebSockets and a web page instead). They perform similar tasks in very different ways. The now-deprecated Karma E2E tests don't use WebDriver and suffer for it - they must maintain state within the browser (WebDriver state is entirely outside of the browser process).
– Andy
4 Answers
103
votes
Not recommended by the current maintainer of Protractor:
https://github.com/angular/protractor/issues/9#issuecomment-19927049
Protractor and Karma should not be used together; instead they provide separate systems for running tests. Protractor and Karma cover different aspects of testing - Karma is intended mostly for unit tests, while Protractor should be used for end to end testing.
Protractor is built on top of WebDriverJS, which uses a Selenium/WebDriver server to provision browsers and drive test execution. Examples of pure WebDriverJS can be found here: http://code.google.com/p/selenium/wiki/WebDriverJs
And
https://github.com/angular/protractor/issues/9#issuecomment-19931154
Georgios - I think it makes sense to keep Protractor and Karma separate - for end to end tests, you want the native event driving and flexibility of webdriver, while for unit tests you want fast execution and autowatching of files.
75
votes
UPDATE. Here is a simple package I've created to add minimal Karma setup to any project with one single command npm install min-karma.
I'd like to clarify some possible misconceptions about Karma and Protractor. Karma FAQ actually does refer to Adapter for Angular's Scenario Runner, which, however, seems to be abandoned, with Protractor being recommended instead.
Karma
Karma is a test runner that will run the JavaScript files specified in you configuration file explicitly or using node-globs. (For non-JavaScript external templates, Angular's Unit Testing Guide recommends using Karma html preprocessor to compile them into JavaScript first.)
These can be all your source files, some of them, some of them plus some additional files or files irrelevant to your project, only providing some extra configuration - you name it! You can have multiple karma config files for different purposes, which you can run in parallel or one-by-one. Each karma process launches its own set of browsers (these are currently available).
This feature of Karma to run only a set of files is what makes it perfect for fast tests running in background upon each source file edit, and get immediate feedback, which is brilliant! The only negative is the "noisy" error reporting that will hopefully improve!
Karma is not only for unit tests
Unit test is for a single unit of your source code. In Angular's case a typical unit is Angular Component (Service, Factory, Provider, Controller, Filter, Directive etc). Remember to keep your Controllers thin, so too many unit tests for latters is a red flag.
In a unit test, every other units of code, on which this unit depends (so-called unit's dependencies) should not be tested at the same time. Instead they should be "mocked", e.g. replaced by something simple like dummy instances. Angular provides great mock environment support. Ideally you want to see all those mocks directly inside your tests, so you never need to wonder where all those dependencies come from.
Karma is just as useful for Integration Tests, where a group of source code units is tested together, with only some of their dependencies being mocked. It is important to remember that any dependency is by default provided from your source code modules (as long as those modules either injected directly in your tests, or they are dependencies of other modules injected (in which case you don't need to inject them, but no harm to do so). The mocked dependencies will override the provided ones.
Running Fast and Frequent is the main feature of Karma. This means you want to avoid any server requests, any database queries, anything that can take longer than fractions of seconds. (Otherwise it will NOT be fast!) Those long processes are ones you want to mock. This also explains why it is a bad practice to put raw low level services like $http directly inside your controllers or any complicated business logic units. By wrapping those low level outside communication services into smaller dedicated services, you make it much easier to "mock them away".
What Karma does not do is running your site as it is, which is what End-to-End (E2E) testing is. In principle, you could use Angular's internal methods to recreate the site or its pieces. Which, for small pieces, can be useful, and a fast way e.g. to test directives.
It is, however, not recommended way to throw complicated code inside your tests. The more you do it, the more chance is that you make errors in that code instead of what you are actually testing.
That is why I personally dislike the often mentioned complicated way of testing methods using low level methods like $http. It works cleaner to isolate any reference to low level methods into dedicated methods of your own, whose single responsibility is to make http requests. These dedicated methods should be able to work with real backend, not a fake one! Which you can easily test - manually or even perfectly fine with Karma running with another special config, as long as you don't mix that config with the one usually used to run Karma regular and fast.
Now, having your dedicated small services tested, you can safely and easily mock them to test your other logic and put these tests into your regular Karma setup.
To summarize. Use Karma to run any set of JavaScript files. It is (should be) fast. You don't see your complete app, so can't test the final result effectively and reliably. Would I run it with Protractor? Why would I? Running Protractor would slow down my tests, defeating the purpose of Karma. It is easy to run Protractor separately.
Protractor
Protractor is:
an end-to-end test framework for AngularJS applications. Protractor runs tests against your application running in a real browser, interacting with it as a user would.
So Protractor does exactly what Karma doesn't - run your real final application. This reveals its both power and limitations:
Running complete application is the only reliable final test that your application works as expected. You can write up complete user story scenarios and put them into your tests!
But it is harder to track errors without isolating individual units of your source code. This is why you still need Karma to test your JavaScript code first.
Now would I want to run Protractor with Karma? I surely can run them in separate terminal windows, in parallel. I could, in principle, have them share test files if I need to, but normally I'd rather not. Why? Because I want to keep my tests small with single dedicated purpose.
The only exception would be a file defining testing macros useful for both runners. This, however, would not be a test file but a macro definition file.
Other than that, I like a clear separation between my tests. Those to be run frequently and fast, and those for the complete app. That makes a clear separation between when using Karma and when Protractor.
1
votes
Karma is a test runner provided by the Angular team, Karma will execute your tests in multiple browsers which shall ensure that our application is compatible in all browsers. Unit Test for angular js can be used karma + jasmine
Jasmine is a javascript unit testing framework and will provide us with utilities to test our application. This works best on the Angular framework and thus, our choice of “automated unit testing tool”. https://github.com/shahing/testingangularjs
And Protractor is an end-to-end test framework for Angular and AngularJS applications. Protractor runs tests against your application running in a real browser, headless browsers , cross browser testing and can be hosted on saucelabs.
