Why is test automation only partially successful with Cypress and Azure DevOps?

Question

I am using Cypress.io (Version 5.1.0) for testing my project. My project is in azure DevOps. Now i want to include my cypress tests in Azure DevOps so my tests will run automatically.

I set up the JUnit reporter on my Cypress project: into my “package.json” file i added

"cypress-multi-reporters": "^1.2.4",
"mocha-junit-reporter": "^1.23.3"

then run npm install

than added

  "scripts": {
  "scripts": "cypress run",
  "test": "npm run scripts"
}

Into my “cypress.json” file i added

    "reporter": "mocha-junit-reporter",
"reporterOptions": {
  "mochaFile": "cypress/reports/junit/test-results.[hash].xml",
  "testsuitesTitle": false
}

After this I created a new Pipeline using Azure Repos in Azure DevOps. For Pipeline Configuration i selected Node.js.

Now I have a YAML file. Here i removed npm build from the first script.

Now I picked npm from the assisstant. On the npm configurations, I selected custom and write the command run test . Now I Select the result format as “JUnit” and set Test results files to “*.xml” At last I selected the option "Merge test results". Now I saved and run the pipeline.

This is what my Job does:

Pool: Azure Pipelines
Image: ubuntu-latest
Agent: Hosted Agent
Started: Yesterday at 17:31
Expanded: Object
Result: True
Evaluating: not(containsValue(job['steps']['*']['task']['id'], '6d15af64-176c-496d-b583-fd2ae21d4df4'))
Expanded: not(containsValue(Object, '6d15af64-176c-496d-b583-fd2ae21d4df4'))
Result: True
Evaluating: resources['repositories']['self']['checkoutOptions']
Result: Object
Finished evaluating template 'system-pre-steps.yml'
********************************************************************************
Template and static variable resolution complete. Final runtime YAML document:
steps:
- task: 6d15af64-176c-496d-b583-fd2ae21d4df4@1
  inputs:
    repository: self


  MaxConcurrency: 0

What is wrong with my automation? How can I fix this?

Update: Thats my yml file:

 # Node.js
# Build a general Node.js project with npm.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/javascript

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: NodeTool@0
  inputs:
    versionSpec: '10.x'
  displayName: 'Install Node.js'

- script: |
    npm install
  displayName: 'npm install'

- task: Npm@1
  inputs:
    command: 'custom'
    customCommand: 'run test'
  continueOnError: true

- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '*.xml'
    searchFolder: '$(System.DefaultWorkingDirectory)/cypress/reports/junit'
    mergeTestResults: true
    testRunTitle: 'Publish Test Results'

I got an email with this Details

Job 1 error(s), 1 warning(s) Error: Npm failed with return code: 254

Please check if the answer below solves your issue. If not, could you edit your question and provide your yaml file? So that we can further investigate the issue. — Jane Ma-MSFT
this did not work for me. I changes ubuntu-latest to ubuntu-18.04 — ABC123
Please try to change the organization. I just found a relative question. — Jane Ma-MSFT
how to change the organization? in cypress or in azure? and if azure, should I add the project? because i want cypress tests to start automatically if there is any update at the project — ABC123
In Azure DevOps, change to another organization and create a new project. Repeating all the configurations of the original project in the new project, just like moving the original project to the new organization. — Jane Ma-MSFT

Jane Ma-MSFT Jane Ma-MSFT · Accepted Answer · 2020-09-09T02:41:26

The issue may be due to the agent rather than your code and scripts.

You can try the following solutions:

Change your agent image. As you are currently using the ubuntu-latest, it is recommanded to try the ubuntu-20.04 or ubuntu-16.04.
Use a self-hosted agent. If you don't have a self-hosted agent, click Self-hosted Linux agent for detailed steps.
Change the orgnization. Choose another organization that can run the build correctly, and just in case, it is better to create a new organization. Then create a new project and try your tests.

Why is test automation only partially successful with Cypress and Azure DevOps?

2 Answers