xunit InlineData tests are not found from CLI

Question

I have found an issue with VSTS Test step where tests are rerun on failure.

If a test fails with "InLineData", I found that it's not found with the test case filter.

Error Message

[xUnit.net 00:00:00.90] HGEM.Hub.UI.Tests: Exception filtering tests: Incorrect format for TestCaseFilter Missing Operator '|' or '&'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed. No test matches the given testcase filter FullyQualifiedName=HGEM.Hub.UI.Tests.Features.GuestJourney.Tests.GuestJourney.BarometerScoreFilterArea(area: West) in C:\Users\brian.mitchell.TMDC\source\repos\HGEM.Hub.UI.Tests\HGEM.Hub.UI.Tests\bin\Run\netcoreapp2.2\HGEM.Hub.UI.Tests.dll

This does not work

dotnet test --filter "FullyQualifiedName=HGEM.Hub.UI.Tests.Features.GuestJourney.Tests.GuestJourney.BarometerScoreFilterArea(area: \"West\")"

This does work

dotnet test --filter "FullyQualifiedName=HGEM.Hub.UI.Tests.Features.GuestJourney.Tests.GuestJourney.BarometerScoreFilterArea

But this will run ever variation of this test which looks like the following

    [Theory]
    [Trait("Category", "GuestJourney")]
    [InlineData("East")]
    [InlineData("North")]
    [InlineData("Northern Ireland")]
    [InlineData("Scotland")]
    [InlineData("South")]
    [InlineData("Wales")]
    [InlineData("West")]
    public void BarometerScoreFilterArea(string area)
    {
        this.Given(_ => _navigationSteps.WhenIClickOn("GuestJourney"))
            .When(_=>_common.WhenIFilterDatesBy("Current Period"))
            .And(_=>_common.WhenIFilterAreaBy(area))
            .Then(_=>_common.ThenISeeABarometerScore())
            .BDDfy<GuestJourneyStory>("Barometer Score filter area");
    }

So my question is this a bug in VSTS? Am i missing some configuration to deal with the rerun of "Theory" Tests?

Levi Lu-MSFT Levi Lu-MSFT · Accepted Answer · 2019-09-23T06:04:32

It seems like an open issue of dotnet test currently. A similar issue was reported and tracked here

xunit InlineData tests are not found from CLI

1 Answers