Azure DevOps "Visual Studio Test" - Cannot Enable "Rerun Failed Tests" When using "Override Parameters"

4
votes

I have a release definition where I want to use both the "Override Parameters" and "Rerun Failed Tests" in the same step, this looks something like this:

enter image description here

enter image description here

When I have just the override parameters specified everything work fine no issue. When I enable the "Rerun Failed Tests" option though I get the following error in my release:

SettingsUpdation.Run : Exception occurred during the updation of run settings: System.FormatException: Error encountered while overriding test run parameters. Please check the test run parameteres provided. Error occurred while publishing test results : Skipping marking test run as complete as test run id 0 is invalid. Error: C:\TFSAgent4_work_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.143.8\Modules\DTAExecutionHost.exe failed with return code: 1 Vstest failed with error. Check logs for failures. There might be failed tests.

In detail it looks like this:

2019-04-04T13:52:40.2481161Z ##[warning]Unable to parse the override run parameters string: -SpecificDatabaseConnectionName=XXX_TestCI -SpecificDatabaseConnectionIPAddress=10.0.0.119 2019-04-04T13:52:40.2941407Z ##[error]SettingsUpdation.Run : Exception occurred during the updation of run settings: System.FormatException: Error encountered while overriding test run parameters. Please check the test run parameteres provided.

I'm not sure what's causing this, it just seems like a bug to me, is there anything else I need to set to make these two options work together correctly?

Update 1

My understanding is that other users have experienced this issue too(per deleted answer), I never tested the previous version of TFS so I don't know if it's specific to Azure DevOps or covers other versions of TFS. I also don't know if this is an issue in Azure's hosted version of DevOps, I'd be curious if that was affected too?

It could be a parameter in the test step, maybe the version of Visual Studio installed on the build agent server, or something else covering the way I run my tests, I've seen allot of things that can cause the build agents to go haywire in the past, I'll continue trying different variations to see if I can determine if any of these are the root cause.

1
azuretestingazure-devopsreleaseazure-pipelines-release-pipeline
Is it failing on the first run, or during the re-run after a test failure? Did you try to run the build with system.debug = true? - DenverDev
@DenverDev Immediately on the first run, before running any tests, I'll enable that option and include the feedback in a update. - David Rogers
@DenverDev It's not a build but a release, don't think you can set that parameter for a release, maybe if I setup a build with that step I could do that, I'll look into it... - David Rogers
It's possible, just a little more complicated. docs.microsoft.com/en-us/azure/devops/pipelines/release/… It may not help, but there is some debug logging right after the task tries to read the override parameters. I was hoping it would help. - DenverDev

1 Answers

3
votes

I think I have this figured out, there seems to be confusion about how I supply my parameters, with a "=" or not, if you see the tool tip:

enter image description here

No "=", now google around a bit and you'll see people using them. Interesting, I've used both before without issue, however playing around with this, I tried switching to the other format:

enter image description here

Seems to resolve the issue, not sure why though...