5
votes

I am new to both Azure DevOps and Maven.

We have set up an Azure build pipeline such that it will deploy artifacts for snapshot builds and also for releases.

I want deployment of release artifacts to be idempotent. That is if the artifact has already been deployed it should not be an error.

The problem is I get a 409 "resource conflict"

Q Is there a way to tell maven to deploy only if the artifact does not exist and that it is not an error if it does.

Is there anyway to do this from DevOps?

For my own education, I would also like to know how to do this for maven (without Azure). This could be via either a command line switch, the pom.xml or the maven settings.xml

It seems to be implied that there is not, if so it is a surprising omission. I would like to understand the rationale.

Bonus points if there is a way to check that the artifact deployed is actually the same as the one just built by the pipeline.

The relevant pipeline snippet is:

   task: Maven@3
#          condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
          inputs:
            mavenPomFile: 'pom.xml'
            options: '-B -s $(mvnSettings.secureFilePath) -DWHERE="AzureDevops" clean deploy'
            mavenAuthenticateFeed: true
            publishJUnitResults: true
            testResultsFiles: '**/TEST-*.xml'

For background this is what I know about Azure and Maven. If I have misunderstood anything it may be a contributing factor.

Maven allow you to deploy two kinds of artifact:

SNAPSHOTS

  • A snapshot is a development version of a package.
  • Snapshots have the suffix -SNAPSHOT. E.g. 1.2.0-SNAPSHOT
  • SNAPSHOTS are mutable. A deploy operation can replace a SNAPSHOT with a new version (more recent development versions can replace development versions)

RELEASES

  • Any version not ending in the suffix -SNAPSHOT is considered a release version.
  • Releases are immutable. A deploy operation will fail if the release has already been deployed to the repository.

Both Azure and Maven considered published artifacts as immutable. Azure understands -SNAPSHOT when acting as a maven repository and allow development versions to be overwritten. The idea is that you cannot (or at least not easily) replace a published artifact that something else might depend upon.

409 = Resource Conflict

This can mean:

  • The artifact has already been published and cannot be overwritten

  • The artifact could not be published because it was the wrong type. For example publishing a release to a repository that only accepts snapshots or publishing a snapshot to a repository that only accepts releases

I am not sure how to tell maven that its okay for the deployment to fail if the artifact already exists. The obvious and wrong hack (in Linux) is:

mvn deploy || /bin/true

This is bad because it will report the deployment step as successful if it has failed for another reason.

There is a maven plugin (https://github.com/chonton/exists-maven-plugin) for doing this. I am not sure how you would use this in Azure. Is this plugin a defacto standard?

See also:


Update 23/06/2020

I am nearly there with this but stuck:

    variables: 
    - name: artifactDoesNotExist
      value: '0'
    - name: mavenRepoURL
      value: 'https://blahblah.visualstudio.com/_packaging/myazurefeedname/maven/v1/com/mycompany/myproject'

    - task: Bash@3
      displayName: 'Check if Maven artifact exists'
      inputs:
        targetType: inline
        failOnStderr: false
        script: |
               #set variable iff artifact exists
               VERSION=`cat VERSION.MVN`; mvn -X -B -s $(mvnSettings.secureFilePath) -DWHERE="AzureDevops" -DremoteRepositories=$(mavenRepoUrl) dependency:get -Dartifact=com.mycompany.project:artifiactId:"$VERSION"
            echo "##vso[task.setvariable variable=artifactDoesNotExist]$?"

    - task: Bash@3
      condition: and(succeeded(), eq(variables['artifactDoesNotExist'], '0'))
      inputs:
        targetType: inline
        script: |
            echo artifactDoesNotExist == 0 -> true

    - task: Bash@3
      condition: and(succeeded(), eq(variables['artifactDoesNotExist'], '1'))
      inputs:
        targetType: inline
        script: |
            echo artifactDoesNotExist == 1 -> true

I suspect the dependency:get command line may not be quite right.

Note: when testing the command I have to remember to delete the artifact from ~/.m2/repository as it look in the local one.

Another strange thing is occurring. Although I have deployed new test versions of the artifact they do not appear in the relevant Azure feed. And yet the first attempted upload succeeds while subsequent uploads fail. Where are these uploads going and why can't I see them in Dev Ops?

The version for which I discovered this issue is still in the feed as a maven artifact 'com.mycompany.myproject:artifactId' with a version.

See also What are the equivalent maven commands and settings to upload and download azure artifacts?

1
What I don't understand yet: Why do you try to deploy release versions again? I mean, if you have already build and deployed 4.0.9, why build and deploy 4.0.9 again? Why is it important how Maven handles this case, why not just don't do it?J Fabian Meier
When the pipeline is run it deploys a new version. If that version does not exist the pipeline fails. That should actually be a null op if the version already exists (and ideally if the checksums match).Bruce Adams
But when you build a release version, you afterwards increase the version number. How can it happen that the pipeline builds the same version as before?J Fabian Meier
The pipeline can be triggered by several things. For example, if there is a merge to the master branch or it is run manually. In one case they are deliberately identical and in another they may not be. Maybe someone merged a documentation change.Bruce Adams
I would suggest that either the release process itself increases the version number (as in the maven release plugin) or that you immediately fail the pipeline if the version it is trying to build already exists. I don't see the benefit of running the pipeline up to the end to then discover that the version you are trying to build already exists.J Fabian Meier

1 Answers

3
votes

This should more related to Maven , there is nothing specific to configure in Azure DevOps side.

You could try to use a command line task in your build pipeline to check first if that release version exists:

mvn dependency:get -Dartifact=g:a:v -o -DrepoUrl=file://path/to/your/repo

More details take a look at this How to determine if a Maven artifact is in my repo from command line?

If that gave (group-artifact-version) does exists, then you don't proceed with the rest of the build.