My project (mobile application) has Gitlab pipeline with stage for running UI tests (Appium) based on TestNG and maven. with surefire plugin. I set up displaying JUnit reports in pipeline results as artifacts artifacts:reports:junit: - surefire-reports/TEST-*.xml and it is correctly displayed on pipeline page. But even if there are failed tests in report, job is still marked as succeeded, and pipeline is green. Is it possible to fail pipeline if report contains failed tests?
0
votes
1 Answers
0
votes
It seems Gitlab will only fail the pipeline if the test process exits with a non zero code, and it won't parse the JUnit xml files to determine the success of a pipeline (https://forum.gitlab.com/t/how-to-fail-job-and-pipeline-if-there-are-failed-tests-in-junit-report/37039/3)
Here is what I did to address this issue:
- Run your test and collect the results in a file
# the echo command will only be run when the test process exits with a non zero code,
# and the entire step will still return 0 as the exit code
run_your_test_for_example_mvn_verify || echo "there is Maven test failure" >> failures.log
run_your_test_for_example_poetry_pytest || echo "there is pytest failure" >> failures.log
- Test the presence of this result file later
# if the failures.log file exists,
# the command below will return false and will fail the pipeline
test ! -f failures.log
