Running testcafe in gitlab CI fails

5
votes

I am trying to run my end to end tests (using testcafe) during the CI pipeline of gitlab. However, I am running into the following error:

ERROR The Firefox 52.0.0 / Linux 0.0.0 browser disconnected. This problem may appear when a browser hangs or is closed, or due to network issues.

My .gitlab-ci.yml is as follows:

stages:
  - test

before_script:
    - apt-get update -yqqq
    - apt-get install -y xvfb
    - apt-get install iceweasel -yqq
    - Xvfb :99 -ac &
    - export DISPLAY=:99

test-frontend:  
  image: node:7.7.4
  stage: test
  script: 
    - npm install
    - npm install -g [email protected]
    - testcafe --list-browsers
    - testcafe firefox e2etests/tests/login.test.js
  tags:
    - vue

So basically I am using a node docker image for my test 'stage' and install xvfb to 'display' a browser.

Output ci gitlab:

npm info ok 
$ testcafe --list-browsers
Using locally installed version of TestCafe.
firefox
$ testcafe firefox e2etests/tests/login.test.js
Using locally installed version of TestCafe.
 Running tests in:
 - Firefox 52.0.0 / Linux 0.0.0

 Try to
ERROR The Firefox 52.0.0 / Linux 0.0.0 browser disconnected. This problem may appear when a browser hangs or is closed, or due to network issues.
2
testingcontinuous-integrationautomated-testsgitlabtestcafe
Unfortunately, it's very hard to diagnose what causes the problem. This error appears when a browser doesn't respond to the TestCafe server for 2 minutes. In most cases, this means that the browser has crashed or hung completely. Would you please provide us with a link to your web site or emulate this problem with a brand new project? – Marion
In addition, try using latest TestCafe version: 0.21.0. – Marion
@Marion It seems to be a gitlab / Ci issue, when I run the specific test locally it is working – M. Suurland

2 Answers

1
votes

I started with the similar approach couple of days ago. But I quickly realised that Puppeteer; node API to manage headless-chrome is what I was looking for, the most painless and best possible way to integrate TestCafe in Gitlab CI.

Lot of resources related to this issue in Google, written by tech-savvy, Kanya West of development world goes above my head (because I am stupid developer who understand only simple code).

Here is how I achieved my goals:

npm install testcafe-browser-provider-puppeteer --save-dev

npm install testcafe --save-dev

Here is gitlab-ci

test_e2e_testcafe:
  stage: test
  image: alekzonder/puppeteer
  script:
    - cd app
    - npm install
    - npm start &
    - ./node_modules/.bin/testcafe puppeteer:no_sandbox path-to-test-folder/yourtestfile.js
  except:
    - master
  tags:
    - autoscale
1
votes

To run Firefox, please also define dbus:

  - Xvfb :99 -ac & 
  - export $(dbus-launch)

Update:

Add the following command before Xvfb:

  - apt-get install -y dbus-x11

In addition, try the config below. I have checked it on gitlab and it works properly for me:

  stages:
    - test

  before_script:
      - apt-get update -yqqq
      - apt-get install -yqq xvfb
      - apt-get install iceweasel -yqq
      - apt-get install dbus-x11 -yqq
      - Xvfb :99 -screen 0 1280x720x24 -ac &
      - export DISPLAY=:99
      - export $(dbus-launch)

  test-frontend:  
    image: node:7.7.4
    stage: test
    script: 
      - npm install
      - npm install -g testcafe
      - testcafe --list-browsers
      - testcafe firefox e2etests/tests/login.test.js
    tags:
      - vue

See the full tutorial here: Integrate TestCafe with GitLab