Ruby on Rails test coverage with simplecov

11
votes


I want to analyse the test coverage of our code , and therefore, installed the simplecov gem.

Our testing environement has 2 seperate project: REST API test (Java+Rest-Assured) and Web UI testing (Java-Selenium).
As you can see, we dont have unit testing inside the rails app, and we are testing using external projects.

I configured the simplecov gem as descriped in the tutorial and put this, in the rails script:

require 'simplecov'
SimpleCov.start 'rails'
puts "require simplecov"

When loading the app, I see the string I printed.

I ran both automation test projects, saw their printouts in the rails log, but I don't see any coverage of controllers/models, I see only small precentage of initializtion files of some gems and rails.

I searched the net, and tried putting the code phrase in boot.rb or even puma.rb and it returned the same results.



Any ideas?

EDIT

Nothing helped with all the comments, but I figured out something very interesting, in all cases, I only see the name of methods marked as tested, not the content (in controllers).
I tried to put the simplecov start phrase in both bin/rails, puma.rb, config.ru, environment.rb, all not given the desired results of code coverage.

3
ruby-on-railsrubyautomated-testssimplecov
There are a couple of things that might go wrong here. First, you should run puma single-threaded. SimpleCov's multi-threading support is limited. Second, have you tried using SimpleCov's at_exit hook? It will help you find out when and where SimpleCov stop "recording" code execution coverage: github.com/colszowka/simplecov#customizing-exit-behaviour . Third, Ruby's coverage might provide everything you need already, so you don't need SimpleCov in the first place: ruby-doc.org/stdlib-2.5.0/libdoc/coverage/rdoc/Coverage.html I hope you find that helpful. - knugie
Our puma is running single threaded, also, what you shared about the ruby lib seems to be as per class, which can be very tough to control - YogevAbr
I dont understand " and we are testing using external projects. " - Simon Franzen
@SimonFranzen Tests are not rails unit tests, but using external projects like selenium and rest-assured. - YogevAbr
Then I am not sure, how SimpleCov can track those tests and the coverage. - Simon Franzen

3 Answers

2
votes

I'm not sure simplecov can measure the whole rails app coverage... But I googled something that you can attach as a rack middleware:

https://github.com/danmayer/coverband

And it's output is compatible with simplecov. So it looks like it could be useful in your case.

As you mentioned in your question you're using puma. I suspect that, since it's multi-threaded, it spawns a few rails apps and their simplecov output overwrites each other's results. I'd try with the single threaded server like webrick - but this may make your tests slower (depending on how the tests are fired up really) or try the coverband gem.

Also - even if the server is single threaded - I'm not sure if each request would not overwrite simplecov's output.

0
votes

Maybe you have to specify the paths

require 'simplecov'
SimpleCov.start do

  # add_filter '/admin/'

  add_group "Models", "app/models"
  add_group "Controllers", "app/controllers"
  add_group "Lib", "lib/"
  add_group "Helpers", "app/helpers"
end
0
votes

You need to to start SimpleCov before loading any of your files, so put these lines as early as possible in your ruby entrypoint:

require 'simplecov'
SimpleCov.start

You could see an example in one of my repos here: https://github.com/tareksamni/DockUp/blob/master/spec/spec_helper.rb

I do autoload my ruby code after starting SimpleCov. You need to the same as well:

require 'simplecov'
SimpleCov.start

require './autoload'