How to configure sbt test / ScalaTest to only show failures?

20
votes

Is there a way to truncate the test results to only show result text for unit tests only when the unit test has failed? I'm working on a Scala project that has 850 unit tests, and the green text from the successful unit tests makes it difficult to focus on just the failures.

Example of what I'm talking about:

[info] - should have colors
[info] - should not be dead
//.... x 100
[info] - animals should not be rainbows *** FAILED *** 
[info] -"[rainbow]s" was not equal to "[ponie]s" (HappinessSpec.scala:31)

What I would like is something that just shows the failure(s):

[info] - animals should not be rainbows *** FAILED *** 
[info] -"[rainbow]s" was not equal to "[ponie]s" (HappinessSpec.scala:31)

I realize there is the test-quick sbt command, but it's still running 300 successful unit tests in my case when there are only 30 failures.

Something along the lines of this in terms of usage is what I'm looking for:

sbt> ~ test -showOnlyFailures

I would also be happy with something that shows all of the failures at the end of running the unit tests. IIRC, this is how RSpec works in Ruby...

4
scalasbtscalatest

4 Answers

18
votes

After adding the following to build.sbt, scalaTest will show a fail summary after the standart report:

testOptions in Test += Tests.Argument("-oI") 

I - show reminder of failed and canceled tests without stack traces
T - show reminder of failed and canceled tests with short stack traces
G - show reminder of failed and canceled tests with full stack traces

There is also a "drop TestSucceeded events" flag, but I have failed to use it: http://www.scalatest.org/user_guide/using_the_runner

5
votes

Updated answer for Scalatest 3.

Here's the new syntax for your build.sbt file to reprint all the failures at the bottom of the test suite run:

testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-oI")

You might also want to suppress the info notifications, so it's easier to see the failures.

testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-oNCXELOPQRM")

Here's what all the options represent:

  • N - drop TestStarting events
  • C - drop TestSucceeded events
  • X - drop TestIgnored events
  • E - drop TestPending events
  • H - drop SuiteStarting events
  • L - drop SuiteCompleted events
  • O - drop InfoProvided events
  • P - drop ScopeOpened events
  • Q - drop ScopeClosed events
  • R - drop ScopePending events
  • M - drop MarkupProvided events

"-oI" prints the error notifications again at the end of the test suite run whereas "-oNCXELOPQRM" only shows the tests that fail.

See the configuring reporters section on the ScalaTest website for more detail.

4
votes

For those using maven with the scalatest-maven-plugin, you can add this configuration:

<configuration>
  <stdout>I</stdout>
</configuration>
2
votes

You could try the sbt command

last-grep error test

This should show you the errors/failures only.