How to use terraform dynamic block to build multiple front end ip in the az load balancer that leverages public ips for azure availability zones

Question

The issue I'm running into is with regard to the load balancer front end ip so it can use a unique Public IP to reach a different server in a different Availability Zone. I'm using count on creating the Public IPs, but I'm not using count on the load balancer because I don't want a new LB for each server. If i could somehow save the Public IPs to a variable then i could reference them using for_each inside the dynamic block but I can't find a way to do that. Here is the code i have so far, but it can't work as is. There may not be a solution to this problem, which really stinks. BTW I'm using the split function below so it returns a list that the properties need. It's a bit hackish but it does work.

resource "azurerm_public_ip" "pip" {
  count               = "${var.nblinuxvms}"
  name                = "${var.proj_name}-lbpip${count.index}-${var.region}-${var.app_env}"
  location            =  var.region
  resource_group_name = "${azurerm_resource_group.rg.name}"
  allocation_method   = "Static"   #Public IP Standard SKUs require allocation_method to be set to Static
  sku                 = "Standard" #Standard SKU Required for Zones
  domain_name_label   = "${var.proj_name}${count.index}${split("", "${element(["1", "2", "3"], "${count.index}")}")}"
  zones = "${var.avzones}" ? split("", "${element(["1", "2", "3"], "${count.index}")}") : null
}
resource "azurerm_lb" "lb" {
  name                = "externallb"
  location            = "${azurerm_resource_group.rg.location}"
  resource_group_name = "${azurerm_resource_group.rg.name}"
  sku                 = "standard" #standard SKU needed to support zones
 dynamic "frontend_ip_configuration" {
   for_each             = "${azurerm_public_ip.test.*.ip_address}" #this is the problem line. I need a way to store all the IPs in a variable and then iterate through them for each new frontend ip configuration
   content{
     name  = "primary${count.index}" #This name is also important as this is how I'll connect the nat rule down below
     public_ip_address_id = "${azurerm_public_ip.pip.id}"
  }
resource "azurerm_lb_nat_rule" "lbnr" {
  count                          = "${var.nblinuxvms}"
  resource_group_name            = "${azurerm_resource_group.rg.name}"
  loadbalancer_id                = "${azurerm_lb.lb.id}"
  name                           = "SSHHost${count.index}"
  protocol                       = "Tcp"
  frontend_port                  = "${2200 + count.index}"
  backend_port                   = 22
  frontend_ip_configuration_name = "primary${count.index}" #This name needs to match the LB Front End IP Configuartion
}

The frontend_ip_configuration_name needs to match the load balancer name. Dynamic block with for_each seems like the best solution for the particular issue as its not a resource...but I don't see a way to save the public ip to any variables i can reference. If there isn't a solution how are people solving this? By creating a separate LB per Azure availability zone? Since it has to be a standard, not basic LB that seems cost prohibitive. Hopefully I've just missed something. Any help would be greatly appreciated. Note i have only shared the relevent code from my terraform project. If more code is needed please let me know.(I couldn't add dynamic block to the question tag because my rep is to low.) Thanks, -Sam Kachar

Answer 1

I feel what you are trying to do makes perfect sense and should be possible. Some things to note:

• Within the for_each block, you can use each.value to access the value (in this case that's the public ip object). See for_each documentation for details
• ip_address refers to the actual assign ip, not to the object
• count is not available in for_each blocks, so the name of the frontend_ip_configuration blocks should be deduced from the public ip object directly.

Given the above, you could try something like this (not tested out!):

resource "azurerm_lb_nat_rule" "lbnr" {
  count                          = "${var.nblinuxvms}"
  ...
  frontend_ip_configuration_name = "config_${azurerm_public_ip[count].name}"
}
dynamic "frontend_ip_configuration" {
   for_each = "${azurerm_public_ip.pip}"
   content{
     name                 = "config_${each.value.name}" 
     public_ip_address_id = "${each.value.id}"
  }

I assume you are using terraform 0.12 given for_each was not available in 0.11. The verbose interpolation syntax you are using has already been deprecated in the latest version, better use the new one:

resource "azurerm_lb_nat_rule" "lbnr" {
  count                          = var.nblinuxvms
  ...
  frontend_ip_configuration_name = "config_${azurerm_public_ip[count].name}"
}
dynamic "frontend_ip_configuration" {
   for_each = azurerm_public_ip.pip
   content{
     name                 = "config_${each.value.name}" 
     public_ip_address_id = each.value.id
}

Answer 2

Wesley, I can't thank you enough for your response. It gave me the confirmation I needed to see this through and that i was in fact heading in the right direction. I just last night was able to get around to testing the solution. It took a little more research to finally get it working though.

Attempting to use the each.value reference kept failing. It was throwing an error something like each.value needs to be used with for_each...Which i found frustrating as for_each was just 2 lines above where i was trying to use/reference it. Moreover in the error it had created 3 error messages, so it was iterating using the for_each. For whatever reason though it couldn't pull values using each.value.

The code that ended up working for my LB / Azure Availability Zones was the following (hint, had to use the iterator option):

resource "azurerm_lb" "lb" {
  name                = "externallb"
  location            = "${azurerm_resource_group.rg.location}"
  resource_group_name = "${azurerm_resource_group.rg.name}"
  sku                 = "standard" #standard SKU needed to support zones
  dynamic "frontend_ip_configuration" {
    iterator = pub
    for_each = azurerm_public_ip.pip 
    content {
      name                 = "config_${pub.value.ip_address}"
      public_ip_address_id = pub.value.id
    }

  }
}

....

resource "azurerm_lb_nat_rule" "lbnr" {
  count                          = "${var.nblinuxvms}"
  resource_group_name            = "${azurerm_resource_group.rg.name}"
  loadbalancer_id                = "${azurerm_lb.lb.id}"
  name                           = "SSHHost${count.index}"
  protocol                       = "Tcp"
  frontend_port                  = "${2200 + count.index}"
  backend_port                   = 22
  frontend_ip_configuration_name = "config_${azurerm_public_ip.pip[count.index].ip_address}"
}

From how i understand the iterator option, you don't have to use it, and can just reference whats in the dynamic block label directly as a prefix but that would be super long and unwieldy. The code example i posted above is functioning code :-). I was very happy when that went through. Spent a few days trying to get this all sorted out.

As for your last statement about this not being version 11 and I don't need all the interpolation syntax, I'm working on cleaning that up. That's one of my tasks on my to do list that I need to complete yet, as i finish up this project for a fully featured VM module. Would have saved me a ton of time had the terraform registry had Azure Zones in their compute module, but in building out all this code I"ve had to learn the language much better than i would've just calling the registry.

Like I said above, thanks again for posting an answer, it confirmed i was headed down the right path. Hope all this time i put in helps someone else out should they run into a similar problem. Just note for anyone who reads this I have a azurerm_public_ip block that creates as many PIP as needed for the VM's and LB. If anyone wants me to add that code i can. Just shoot me a message or comment on my post.

Cheers, -Sam Kachar