Azure data factory. Use many different git branches

4
votes

My coworker and I are working in two different git branches: mybranch and hisbranch. Each of us is developing a databrick's notebook for data processing. This notebooks are then added as an activity in two different ADF's pipeline.

For testing the pipeline It's needed to be publish, but when I push it to the repo an error message pop's up: Publish is only allowed from collaboration ('develop') branch.

That forces us to work in the very same git branch (the collaboration one) which is unconvinient because we may conflict a lot during the development and testing.

Is there any way to test pipelines in independent branches and then merge?

1
gitazureazure-data-factory
Do you definitely need to publish? Usually you can test your changes by using the "Debug" button on the toolbar at the top of the pipeline you have made changes to. – Claire Furney
I'd like a programatic way to create an adf_publish from any branch and deploy it. – Ana Lívia

1 Answers

2
votes

Note: Your Azure Repos collaboration branch that is used for publishing. By default, its master. Change this setting in case you want to publish resources from another branch.

To develop multiple features in parallel, you need to just use "Save". Save will commit your changes to the branch you are actually working on. Other branches will do the same. Whenever you want to publish, you need to first make a pull request from your branch to master, then publish. Any merge conflict should be solved when merging everything in the master branch. Then just publish and there shouldn't be any conflicts, and aDF_publish will get generated after that.

For more details, refer "Source control in Azure Data Factory".

Hope this helps.