How can I copy whole schema/database in Azure Data Factory pipeline?

0
votes

I need to copy one database from my PostgreSQL on Azure. I want to do this in ADF. In copy data activity when I specify source and target database, ADF want me to specify also table name. It doesn't make any sense when I want to copy database that have around 100 tables. I have tried wildcard * in dynamic conditions and I was trying to leave tablename blank.

Do I really need to create each copydata activity for each table ?

1
azureazure-data-factoryazure-data-factory-pipelineazure-postgresql

1 Answers

0
votes

I have found a solutions for this question.

We need two pipelines (get_tables and copy_tables)

First create get_tables pipeline. Select all tables from your DB by lookup query

select table_schema,table_name from information_schema.tables where table_schema = 'schema'

After this move output to copy_tables pipeline.

In copy_tables pipeline create for_each with activity of copy data.

By parameters we can transfer table names to each activity.