Azure Data Factory - different copy data mappings views in the same pipeline?

0
votes

I am trying to set up a pipeline with copy data activity in Azure Data Factory and I am confused by the different view of mapping in the copy activity. I have created the pipeline from the template "Copy data from on premise SQL Server to SQL Azure" and I am cloning the activity so there shouldn't be any differences. The source is the same in both activities and I use query against the source database.

Here's how I see it:

Original copy activity:

enter image description here

Cloned copy activity:

enter image description here

I would like to understand why I see different views of mapping.

Thanks in advance!

1
azureazure-data-factory
The source may be the same, but if the sinks are different then the mapping will be different. Please post screenshots of the sink tabs of each copy activity. – JeffRamos
Hi @Ruslan, If my answer is helpful for you, hope you can accept it as answer( click on the check mark beside the answer to toggle it from greyed out to filled in.). This can be beneficial to other community members. Thank you. – Leon Yue
@JeffRamos Yes, the sinks are different from the table perspective, but the destination database is the same. I have figured it out - I was using the query that contained the "count(*)" aggregate. Removing it and the "group by" clause made the mapping view the same as for the original Copy activity. – Ruslan
Hi @Ruslan, do you have any other concerns? – Leon Yue

1 Answers

0
votes

I don't that's an problem. When we clone a copy active, before we debug or run the pipeline, we need to check all the settings manually.

From your screenshots, original copy activity miss the source schema during mapping. Just import schema will be solved it. enter image description here

And the cloned copy active seams import schema automatically. Not sure if all the columns are mapped(I think not). Some suggestions:

  1. Please import the schema in source dataset firstly, and fully set one copy active.
  2. Then clone the copy active which may avoid the problem.

Data Factory may not very smartly and even we clone active, we still should check all the settings in each actives.

Update:

Like @JeffRamos said, sinks are different then the mapping will be different.

We are glad to hear that you have figured it out:

  • "I have figured it out - I was using the query that contained the "count(*)" aggregate. Removing it and the "group by" clause made the mapping view the same as for the original Copy activity."

Thanks for @JeffRamos's useful comment again.

HTH.