Azure DevOps pipeline - how to catch error

0
votes

I am using Azure DevOps build pipeline, to run my selenium web automation tests (run by maven, inside docker container)

  1. Now, even If one of my test scenarios fail, pipeline job is still considered as successful, how can I specify to look for certain log to fail it?

The 'pipeline job/task/phase' I am using to run my test is 'docker compose'

  1. My second question is, is there possibility to filter pipeline output logs? Currently it is flooded with output from few services run in container:

enter image description here

The only thing I found is, possibility to search through logs, but no filtering, regards.

3
azuredockercontinuous-integrationazure-devopspipeline

3 Answers

2
votes

If your objective is to fail the build when one of more of your test has failed, I advise you to add one more step to your build process : Publish Test Results task

This is a necessary step for test running with another task than the default Visual Studio Test task, which consist in publishing your test result file to Azure DevOps, and enable your build to be aware of your tests results (and then let you decide what to do if one or more tests fail)

In your case, you will also probably have to find a way to extract test results file from your containers, as your test results might probably be produced and stored inside of your containers (and unavailable to the Publish Test Result task)

For your second question, I am not aware of any way to filter the output logs directly from the web interface, sorry :(

2
votes

We ran into this with our cypress tests (you should ditch selenium for cypress, its sweet) and solved it by grabbing the exit code manually. We also found that AzureDevops will hang if there's a background process running, even if there's an error, so be sure to look out for that as well if you start up your web server like we do.

  - bash: |
      yarn test-ci:e2e  2> /dev/null 
      if [ $? -eq 0 ]
        then
          yarn stop
          exit 0
        else
          yarn stop
          exit 1
      fi
    displayName: 'Run Cypress Tests'
0
votes

For anyone looking for a way to filter logs, if there are multiple services running, you can create new azure build pipeline task (Docker) that runs docker command:

docker logs -f NAME_OF_THE_SERVICE

This way you will only see logs from desired service.