How to run automated Tests in Azure Test Plans

Question

I want to be able to run automated tests from within Azure Test Plans and define everything in a yml file.

What I've done: I have a repository in azure DevOps containing several tests written in .NET core. Those are associated to a testcase in azure test plans. I also have created a azure pipeline by adding a .yml file to my repo. This pipeline consists of two stages, one for the build and one which actually runs all tests on a nightly schedule.

However, I want the manual testers to be able to trigger the automated tests from within Azure Test Plans too. The official docs say one must create a release pipeline to do so. So my questions are:

Why do I need a release pipeline? I already have my build pipeline, I won't do any release, I just want to be able to run my tests from within test plans by button click.
If I really have to create release pipeline, how can i define this in a yml file? The official docs only mention the "Classic", manual steps within azure DevOps, but I don't want to add pipelines manually, I want to have everything defined in yml files.
Is there a possibility to run my tests containerized? I do it already in my build pipeline, but since I couldn't find a way to define release pipelines in yml, I don't know how to run them in docker containers.

Is my answer helpful for your questions? Or do you still have other questions? — Frank Wang-MSFT

Frank Wang-MSFT Frank Wang-MSFT · Accepted Answer · 2019-10-10T10:00:58

Azure Pipelines offers a compelling orchestration workflow to obtain test binaries as artifacts and run tests. This workflow shares the same concepts used in the scheduled testing workflow, meaning users running tests in scheduled workflow will find it easy adapt; for example, by cloning an existing scheduled testing release pipeline. Another major benefit is the availability of a rich set of tasks catalog that enable a range of activates to be performed before and after running tests. Examples include preparing and cleaning test data, creating and cleaning configuration files, and more.
To define release pipeline in yml, you should go to the preview features page and turn on the toggle for “Multi-stage pipelines”. Below is a simple sample.

stages:
- stage: Build
  jobs:
  - job: Build
    pool:
      vmImage: 'Ubuntu-16.04'
    continueOnError: true
    steps:
    - script: echo my first build job
- stage: Deploy
  jobs:
    # track deployments on the environment
  - deployment: DeployWeb
    pool:
      vmImage: 'Ubuntu-16.04'
    # creates an environment if it doesn’t exist
    environment: 'smarthotel-dev'
    strategy:
      # default deployment strategy
      runOnce:
        deploy:
          steps:
          - script: echo my first deployment

More detailed information, you can refer to https://devblogs.microsoft.com/devops/whats-new-with-azure-pipelines/

How to run automated Tests in Azure Test Plans

1 Answers