Terraform - Azure - VNET Peering Conundrum

Question

I have a bit of a chicken and egg issue. Imagine multiple subscriptions, each with their own VNet. I need to peer these all with a hub which is also in its own subscription. The issue is that I am using a separate state file for each subscription.

Issue: I do not have one Terraform file to build the entire environment. I run an apply for each subscription. Which means I would need to run it twice.. Once for the standup for each subscription to get the VNets in place, then once again for the VNet peering after adding the module/resource to the file for peering.

This will probably cause lots of problems. Especially if I run a destroy/re-apply.

Is there a better way of doing this? If it were one file for the entire environment, Terraform would make sure the VNets were there OR I could use a 'depends_on' attribute. But, the way I'm doing this might break everything.

Thoughts:

Do peering OUTSIDE of Terraform
Use a separate Terraform state file for peering - probably a really bad idea
Do a two step process; setup subscription/Vnet in one run and peering in a second - probably won't work very well
Something I'm missing.

Thanks!

Nancy Xiong Nancy Xiong · Accepted Answer · 2020-02-11T01:02:28

If you have deployed each VNet in each subscription, I think you can do a two-step process: setup subscription/Vnet in one run and peering in a second. The terraform peering configuration will like this, use alias for one specific subscription that you will refer to, use data to query the existing resources in each subscription. Ensure that the Service Principal you're using either has permissions to both Subscriptions or a different Service Principal is used for each Provider block (with the associated permissions).

For example,

provider "azurerm" {
  version         = "xxx"
  tenant_id       = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
  subscription_id = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
  client_id       = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
  client_secret   = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
  alias           = "dev"
}

provider "azurerm" {
  version         = "xxx"
  tenant_id       = "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"
  subscription_id = "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"
  client_id       = "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"
  client_secret   = "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"
  alias           = "test"
}

data "azurerm_virtual_network" "dev" {
  name                = "dev-network"
  resource_group_name = "dev-network-rg"
  provider            = "azurerm.dev"
}

data "azurerm_virtual_network" "test" {
  name                = "test-network"
  resource_group_name = "test-network-rg"
  provider            = "azurerm.test"
}

resource "azurerm_virtual_network_peering" "dev-to-test" {
  name                         = "dev-to-test"
  resource_group_name          = "${data.azurerm_virtual_network.test.resource_group_name}"
  virtual_network_name         = "${data.azurerm_virtual_network.test.name}"
  remote_virtual_network_id    = "${data.azurerm_virtual_network.test.id}"
  allow_virtual_network_access = true
  allow_forwarded_traffic      = true
  provider                     = "azurerm.dev"
}

resource "azurerm_virtual_network_peering" "test-to-dev" {
  name                         = "test-to-dev"
  resource_group_name          = "${data.azurerm_virtual_network.dev.resource_group_name}"
  virtual_network_name         = "${data.azurerm_virtual_network.dev.name}"
  remote_virtual_network_id    = "${data.azurerm_virtual_network.dev.id}"
  allow_virtual_network_access = true
  allow_forwarded_traffic      = true
  provider                     = "azurerm.test"
}

If you are using the Azure CLI auth, you could refer to this.

Alternatively, as you thought, you could try to use a depends_on attribute in the azurerm_virtual_network_peering block refer to this example.

resource "azurerm_virtual_network" "spoke1-vnet" {
  provider            = "azurerm.dev"
  name                = "spoke1-vnet"
  location            = azurerm_resource_group.spoke1-vnet-rg.location
  resource_group_name = azurerm_resource_group.spoke1-vnet-rg.name
  address_space       = ["10.1.0.0/16"]

}
resource "azurerm_virtual_network_peering" "spoke1-hub-peer" {
  provider                  = "azurerm.dev"
  name                      = "spoke1-hub-peer"
  resource_group_name       = azurerm_resource_group.spoke1-vnet-rg.name
  virtual_network_name      = azurerm_virtual_network.spoke1-vnet.name
  remote_virtual_network_id = azurerm_virtual_network.hub-vnet.id

  allow_virtual_network_access = true
  allow_forwarded_traffic = true
  allow_gateway_transit   = false
  use_remote_gateways     = true
  depends_on = ["azurerm_virtual_network.spoke1-vnet", "azurerm_virtual_network.hub-vnet" , "azurerm_virtual_network_gateway.hub-vnet-gateway"]
}

For more information, you could refer to this blog1 and blog2 for deploying to multiple subscriptions with terraform.

Terraform - Azure - VNET Peering Conundrum

1 Answers