We are in process of automating the pipeline creation using Python SDK.
Got around 300 tables need to be moved between Oracle and SQL Server, 90% tables are 1-1 copy so we used SDK to create the pipeline programmatically.
There are 10% tables needs complex data transformation in ADF, which we design in UI and export as template.
The question is, how to create pipeline using SDK and exported templates in ADF?
Here is the sample in MSDN about creating a pipeline:
act_name = 'copyBlobtoBlob'
blob_source = BlobSource()
blob_sink = BlobSink()
dsin_ref = DatasetReference(reference_name=ds_name)
dsOut_ref = DatasetReference(reference_name=dsOut_name)
copy_activity = CopyActivity(name=act_name,inputs=[dsin_ref], outputs=[dsOut_ref], source=blob_source, sink=blob_sink)
#Create a pipeline with the copy activity
p_name = 'copyPipeline'
params_for_pipeline = {}
p_obj = PipelineResource(activities=[copy_activity], parameters=params_for_pipeline)
p = adf_client.pipelines.create_or_update(rg_name, df_name, p_name, p_obj)
print_item(p)
And you can exported templates here:
