Share Data Management Gateway across Azure Data Factory Instances

2
votes

I have a 3-TIER setup with DEV/TEST/PROD (but lets ignore TEST for this post). For each TIER I have an ADF instance running in a separate resource group within the same subscription. My DEV-ADF instance contains a Gateway to load local data into an Azure SQL DW - lets call the gateway "MyGateway". I deployed the same ADF project also to my PROD-ADF instance but I cannot create a new Gateway with the same name. Also, I do not want to spin up a second VM to run another DMG there. So right now, my PROD-ADF instance does not have a Gateway and the LinkedServices that reference the Gateway return status "Offline"

HOWEVER, data is loaded correctly on both systems (DEV and PROD)! Both seem to use the same Gateway which only exists in the DEV-ADF instance. I do not have any other explanation why the PROD load would work otherwise. So the question is, whether the Gateways are shared explicitly between ADF instances. Can someone confirm this? if Yes, at which scope are they shared? Subscription? Tenant? all Tenants I have access to?!?

thanks in advance, -gerhard

2
azureazure-data-factory

2 Answers

1
votes

Gateway name currently is unique per Azure Region per subscription. So usage of same gateway name for the same subscription in the same region would not be possible.

Gateway sharing is not officially supported yet.

But technically it is possible to refer to the gateway resource created under 'DEV' Data Factory from your 'PROD' Data Factory, as both might be in same region, and part of same subscription. We do not recommend customers to re-use gateway this way.

We are working on to support this feature.

0
votes

That sounds like very odd behaviour! As I understand it you can "currently" only have only 1 gateway installed on a target machine. Plus if the PROD instance reports offline then wtf!

For info, there is this feedback post regarding multi gateways per single machine already being worked on.

https://feedback.azure.com/forums/270578-data-factory/suggestions/11428356-data-management-gateway-for-multiple-data-factorie

Are you sure that the linked services for prod and dev are different in terms of where the data lands post copy via the gateway?? If you've deployed the same factory code from VS the gateway issue might be a red herring if the other linked services are the same.... Just a thought.

Failing that I would suggest you've uncovered a bug and should report it to the MS peeps via a support ticket in the portal.