Cycle Through External Data Source in terraform

Question

I'm trying to provision n VMs (assume n=5) in terraform.

The catch is that for each IP that gets assigned to the corresponding VM, I first need to invoke an external tool (an .exe). This tool takes 2 parameters: a regex pattern for the IP to be obtained, and the machine name. The tool then reads out one single free IP address to be used based on the supplied regex, from a Sharepoint Excel file that acts as an IPAM (IP address management solution). The tool also updates the Excel file so that it stamps the supplied machine name next to the IP returned.

For a single VM, this works just fine, and everything completes as expected, leveraging one external data source and one resource. The machine name is read from an input file, same as the regex for the IP, which uses its own file. The external tool is then invoked:

data "external" "Machine_name" {
    program = ["cmd.exe", "/c type c:\\Users\\malbert\\Desktop\\Excel2VM\\MachineName.txt"]
}

data "external" "IPregexMatchPattern" {
    program = ["cmd.exe", "/c type c:\\Users\\malbert\\Desktop\\Excel2VM\\IPregexMatchPattern.txt"]
}

output "MachineName" {
  value = "${data.external.Machine_name.result.name}"
}

data "external" "NetworkObtainedData" {
      program = ["cmd.exe", "/c ExcelUpdateTool.exe ${data.external.IPregexMatchPattern.result.IPregex} ${data.external.Machine_name.result.name} available" ]
}

# Use as an output so the user can see the value as well
output "ip" {
    value = "${data.external.NetworkObtainedData.result.ip}"
}
output "netmask" {
    value = "${data.external.NetworkObtainedData.result.netmask}"
}

output "gw" {
    value = "${data.external.NetworkObtainedData.result.gw}"

Up next, the VM is named based on the input file:

resource "vsphere_virtual_machine" "vm" {
  # Name the VM
  name             = "${data.external.Machine_name.result.name}"

  resource_pool_id = "${data.vsphere_resource_pool.pool.id}"
  datastore_id     = "${data.vsphere_datastore.datastore.id}"

However when trying to scale for n VMs, I'm running into the following issue: I can iterate over n items within the resource section, thus spawning n VMs; however I can't figure out how can I do the same for the external data source, since this one doesn't support the constructs designed for loops (count / length(...)). What I've achieved so far follows. A new .tf file was defined, so that it contains all the names of the machines in a list:

variable machineNamesList {
        default = [ "terraform-firstMachine", "terraform-secondMachine"]
}

Under the resource section, the list is cycled through:

resource "vsphere_virtual_machine" "vm" {
  count = "${length(var.machineNamesList)}"
  # We'll name the VM the same as the guest running inside
  name             = "${var.machineNamesList[count.index]}"

  resource_pool_id = "${data.vsphere_resource_pool.pool.id}"
  datastore_id     = "${data.vsphere_datastore.datastore.id}"

But how can I invoke the external tool n times, each time a new VM is processed ?

Any pointer in the right direction would be appreciated.

It would be useful if you could show what you've tried so far with the working code for a single VM and the code that doesn't work for multiple VMs and why it doesn't work (show any errors or explain what it does do and how that differs from what you want). — ydaetskcoR
@ydaetskcoR: You're right - my original post was written in a rush. I've expanded it now, and also inserted pictures (unfortunately not inline, since I don't have the minimum required reputation yet). — Mihai Albert
Please don't include screenshots of code. Instead paste the code into the question as text and format it as code (either through the UI button, indenting everything by 4 spaces or highlighting it all and hitting ctrl k) — ydaetskcoR

Martin Atkins Martin Atkins · Accepted Answer · 2019-01-17T16:31:21

The count argument is supported for all resource and data blocks, so you can achieve this by setting count on the various objects to the same expression:

variable "machine_count" {
}

data "external" "machine_name" {
  count = "${var.machine_count}"

  # (presumably in practice you'll use count.index in here somewhere)
  program = ["cmd.exe", "/c", "type c:\\Users\\malbert\\Desktop\\Excel2VM\\MachineName.txt"]
}

resource "vsphere_virtual_machine" "vm" {
  count = "${var.machine_count}"

  name = "${data.external.machine_name.*.result.name[count.index]}"

  # ...etc...
}

There's more information on this mechanism in the using variables with count section of the Terraform documentation. Although the examples there are talking about resource blocks, the count mechanism works the same for both resource and data blocks.

Cycle Through External Data Source in terraform

1 Answers