Cucumber test suite too slow for Travis

Question

I have a cucumber test suite for my rails app, comprising around 500 Scenarios with about 5000 Steps between them.

I have set up my github repository to use Travis-CI, using the following .travis.yml.

language: ruby
rvm:
  - "1.9.2"
script:
  - RAILS_ENV=test bundle exec rake db:migrate --trace
  - bundle exec cucumber -f progress -r features features/cards/base_game
  - bundle exec cucumber -f progress -r features features/cards/basic_cards
  - bundle exec cucumber -f progress -r features features/cards/intrigue
  - bundle exec cucumber -f progress -r features features/cards/seaside
  - bundle exec cucumber -f progress -r features features/cards/prosperity
  - bundle exec cucumber -f progress -r features features/cards/interactions
before_script:
  - cp config/database.travis.yml config/database.yml
  - psql -c 'create database dominion_test' -U postgres

I have split the cucumber execution up as Travis was throwing Out Of Memory if I just ran bundle exec cucumber to run all the cases.

However, my most recent push spawned a Travis task which took over 50 minutes to run all my tests, and was therefore killed. Am I just being unreasonable with that many scenarios, or is there anything I could do to speed up execution?

Edit: In case it matters, I should clarify that my Scenarios don't test the GUI. They're testing the rules of a card-game server, so they invoke model methods directly.

Chowlett Chowlett · Accepted Answer · 2013-02-28T09:28:05

I found a solution, after much googling, at this page of Travis' docs.

Basically, it's allowed (recommended, even!) to parallelise the runs. With the following .travis.yml, I end up with six concurrent jobs, none of which takes over 15 minutes, and which therefore all run to completion:

language: ruby
rvm:
  - "1.9.2"
env:
  - CARD_SET=base_game
  - CARD_SET=basic_cards
  - CARD_SET=intrigue
  - CARD_SET=seaside
  - CARD_SET=prosperity
  - CARD_SET=interactions
script:
  - RAILS_ENV=test bundle exec rake db:migrate --trace
  - bundle exec cucumber -f progress -r features features/cards/$CARD_SET
before_script:
  - cp config/database.travis.yml config/database.yml
  - psql -c 'create database dominion_test' -U postgres

Cucumber test suite too slow for Travis

1 Answers