Access azure storage account from pipeline agent in same region with access restrictions enabled

Question

We are using an azure storage account for our cloud services. This storage account is part of a virtual network, so access to the storage account is restricted to selected networks and the vnet is added. This works beautifully in our services.

The problem arises when we try to copy data to this storage account in an azure pipeline. Within the pipeline, we temporarily add a firewall rule to the storage account to allow traffic from the pipeline agent's Ip address to the storage account. Then we copy the data (via azcopy) and finally, remove the firewall rule. This works fine on a private agent. However, we are also using private agents hosted in azure. The problem is that if the agent runs in azure, the connection to the storage account uses private azure ip addresses, and the firewall rule doesn't work. This is specified in this doc:

Services deployed in the same region as the storage account use private Azure IP addresses for communication. Thus, you cannot restrict access to specific Azure services based on their public outbound IP address range.

Is there any way to force external routing? It seems really silly to me that with the current configuration, we are unable to connect to the storage account from within azure, and we ARE able to connect from a private agent (or any other pc) outside of azure.

I've already tried to play with the routing preference setting in the firewalls and virtual networks section of the storage account, and using the -internetrouting endpoint, but this doesn't make any difference.

Answer 1

As per here, you need to allow access to the whole azure data center that match your region. So I am thinking of something automated that will query the API, retrieve the IP range and use it.