BadRequest: The document creation or update failed because of invalid reference

Question

I'm trying to deploy an ADF pipeline using CICD (Azure DevOps Deployment) using a release pipeline. Here I'm trying to merge stuff from my Collaboration branch to master (usig GIT). I'm also using:

Azure Deployment: Create Or Update Resource Group action on SiteOpsConsolidatedProd (Agent Job) as a part of the Continuous Deployment Process

I'm getting the following error:

[error]At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details. [error]Details: ##[error]BadRequest: The document creation or update failed because of invalid reference 'irslinked'. ##[error]Check out the troubleshooting guide to see if your issue is addressed: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting ##[error]Task failed while creating or updating the template deployment.

1. irslinked is the name of a self-hosted IR (Linked)
2. this IR: irslinked is also included in the overwrite template parameters list.

Answer 1

As suggested by Microsoft in their ADF CICD best practices, the self-hosted IR should be hosted on a dedicated ADF instance and should be of type shared and linked to the dev/test/prod instances. Link to the documentation: https://docs.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment#best-practices-for-cicd

Please respond if it works.

Answer 2

This may be helpful for someone. I had a similar invalid reference error. Actually, my ADF was imported from git. I have reconfigured the git with "import existing resource to repository" option.

Azure documentation https://docs.microsoft.com/en-us/azure/data-factory/ci-cd-github-troubleshoot-guide#recover-from-a-deleted-data-factory

If customer had a Self-hosted Integration Runtime in deleted ADF, they will have to create a new instance in new ADF, also uninstall and reinstall the instance on their On-prem machine/VM with the new key obtained. After setup of IR is completed, customer will have to change the Linked Service to point to new IR and test the connection or it will fail with error invalid reference.

Answer 3

I had this error message but it referenced a resource that didn't even exist.

I was able to solve it using steps from here: https://docs.microsoft.com/en-us/azure/data-factory/source-control#troubleshooting-git-integration

Notably:

• Disconnect from existing git repo.
• Reconnect back to the same repo, but select "import existing resources" into a "Create new git branch".
• Then use git to create more commits on top of that branch, which remove all the extra crud that got generated, until the codebase is back to the desired state.

Note that I had to have sequential commits deleting stuff in dependency order, before it would work. i.e. Pipelines, then dataflows, then datasets, then linked services, then Int.Runtimes.