How to publish pipeline in Azure Data Factory enabled with DeVOPS GIT repo using .NET Data Factory SDK (C# )?

Question

I’m facing some issue in deploying the pipeline to Data Factory with GIT enabled (DevOps) and need someone help or advise on that.

When I deploy the pipeline through below code snippet its deploying into Data Factory Repo but instead we need to publish the code to Azure DevOps GIT Repo.

Below is a code snippet used to publish pipeline to ADF using .NET Data Factory SDK (C#)

// Authenticate and create a data factory management client

var context = new AuthenticationContext("https://login.windows.net/" + tenantID);
ClientCredential cc = new ClientCredential(applicationId, AuthenticationKey);
AuthenticationResult result = context.AcquireTokenAsync("https://management.azure.com/", cc).Result;
ServiceClientCredentials cred = new TokenCredentials(result.AccessToken);
DataFactoryManagementClient client = new DataFactoryManagementClient(cred) { SubscriptionId = subscriptionId,  };

// Below snippet deploys pipeline into data factory Repo

client.Pipelines.CreateOrUpdate(resourceGroup, dataFactoryName, pipelineName, pipeline);

But now we need to publish this pipeline code to master branch for the DevOps GIT enabled in our ADF.

Below is one code snippet I found but not sure how to set this object to data factory in deploying the pipeline to DevOps GIT.

FactoryRepoConfiguration repo = new FactoryVSTSConfiguration("account Name", "Repo name", "branch name", "/", "project name");

I have gone through many blogs but didn’t find any help on that.

So could someone please help me on this?

Answer 1

If i don't misunderstanding your requirement,you just wanna implement below creation with sdk code as same as portal(This is just sample project,no security for it then):

In fact, you are very close to your goal because you already found FactoryRepoConfiguration Class. Please refer to the document,you could define 2 types of Repo: VSTS and Github.

I tested Github for your reference:

Answer 2

As per Microsoft's feedback there is no SDK which can publish code directly into a GIT branch in ADF.

I have posted an idea in ADF forums to implement the same and below is the link. If anyone have a same requirements request to up vote the posted idea in below link.

https://feedback.azure.com/forums/270578-data-factory/suggestions/40195249-implement-a-net-sdk-to-publish-pipeline-code-to-a

Thanks.

Answer 3

There is another approach to publish ADF, from master (collaboration) branch or even from your local machine (if needed). With that approach, you can develop your ADF using your .NET framework and when all files ready - publish them into ADF service.
Check this out: azure.datafactory.tools (PowerShell module)