Run cucumber scenario again until pass

Question

I have 10 tests that to be run in cucumber and I am using ruby to write my codes. The application under tests design, forces me to have tests that are dependent to each other.

So until the previous case is a pass, the next one fails. I have added cucumber_wants_to_quit in case the second scenario fails, to avoid waiting to only get the failure reports. Now, I was looking for something like, running the second scenario again, and keep running it until pass before proceeding to the third scenario.

I know of the command cucumber re-run but will not want to use it, since this basically runs all the scenarios and runs the failed cases once the first complete run is completed.

My requirement is basically, when the second scenario fails, before moving to third, run the second scenario until pass. In coding language something like

 After do |scenario.name|
     if scenario.name == failed?
         status = scenario.name.run_again until scenario.name == passed
         if !status
            Cucumber.wants_to_quit = true
         end
     end
 end

Any help on this?

If the second scenario fails, how would running it again make it pass? — grizzthedj
Intermittent errors are frequent in the second test.. But when you take it as a general question. Is there a possibility to do such? Though I agree it's not the best practices — Emjey

grizzthedj grizzthedj · Accepted Answer · 2017-05-31T19:34:32

I think you are climbing up a slippery slope having tests that are dependent on each other to pass. You will likely have continuous problems with this approach going forward.

I would recommend writing your tests so that they are not dependent on each other, and use a framework like Factory Girl to stub out test objects etc. There is a lot of flexibility here, and integrates well with cucumber.

UPDATE: As per your comment, the testing of OTP should be an integration test that is done outside your TEST environment. When you deploy new code to your DEV environment for example, your integration test(that tests OTP) should get run on the code deployed.

In your TEST environment, you can fake the OTP(with Factory Girl). This will eliminate the intermittent failures, and make your tests independent of one another, which makes your test suite more clean and maintainable.

In the short term, perhaps you can create a global status variable to hold the status of the test run. The downside is that you would have to check this in every test.

Run cucumber scenario again until pass

1 Answers