Passing Terraform variables between Modules within Modules?

Question

I have an issue I have gotten stuck on building a module from the Kubespray project.

I have the following folder structure:

terraform/
modules/
   kubespray/
      modules/
         compute/
         ips/
         network/
      main.tf
      variables.tf
roles/
    kubespray/
        terraform_setup/
             main.tf

Within the kubespray/main.tf module there are variables being passed around:

module "ips" {
  source = "./modules/ips"

  number_of_k8s_masters         = "${var.number_of_k8s_masters}"
  number_of_k8s_masters_no_etcd = "${var.number_of_k8s_masters_no_etcd}"
  number_of_k8s_nodes           = "${var.number_of_k8s_nodes}"
  floatingip_pool               = "${var.floatingip_pool}"
  number_of_bastions            = "${var.number_of_bastions}"
  external_net                  = "${var.external_net}"
  network_name                  = "${var.network_name}"
  router_id                     = "${module.network.router_id}"
}

module "compute" {
  source = "./modules/compute"
  ...
  k8s_master_fips                              = "${module.ips.k8s_master_fips}"
  k8s_master_no_etcd_fips                      = "${module.ips.k8s_master_no_etcd_fips}"
  k8s_node_fips                                = "${module.ips.k8s_node_fips}"
  bastion_fips                                 = "${module.ips.bastion_fips}"
  ...
}
output "private_subnet_id" {
  value = "${module.network.subnet_id}"
}

output "floating_network_id" {
  value = "${var.external_net}"
}

output "router_id" {
  value = "${module.network.router_id}"
}

output "k8s_master_fips" {
  value = "${concat(module.ips.k8s_master_fips, module.ips.k8s_master_no_etcd_fips)}"
}

output "k8s_node_fips" {
  value = "${module.ips.k8s_node_fips}"
}

output "bastion_fips" {
  value = "${module.ips.bastion_fips}"
}

If I run my terraform init/apply from inside the submodule in modules/kubespray/ It works fine

If I run from my role/kubespray that has

module "kubespray" {
  source    = "../../../modules/kubespray"
  providers = {
    openstack.src = openstack
  }
}

it fails with

Error: Invalid index

on ../../../modules/kubespray/modules/compute/main.tf line 670, in resource "openstack_compute_floatingip_associate_v2" "k8s_node": 670: floating_ip = "${var.k8s_node_fips[count.index]}" |---------------- | count.index is 0 | var.k8s_node_fips is empty list of dynamic

Help would be extremely appreicated

Answer 1

When you run terraform, the directory becomes an implicit "root module". As it's explained in the Input Variables documentation, root modules can access TF inputs from the CLI or environment, but all other called modules need to have these passed in.

I'm assuming what's happening could be that your ~kubespray/variables.tf input variables have default values assigned which are taking effect. If that's the case, then these should be copied over to ~terraform_setup/variables.tf and passed into the calling module:

module "kubespray" {
  source    = "../../../modules/kubespray"
  providers = {
    openstack.src = openstack
  }
}

Answer 2

Without asking for further information, my guess here is that you have an empty array being passed in (${var.k8s_node_fips[count.index]}) with a value greater than 0 for var.node_root_volume_size_in_gb and var.number_of_k8s_nodes_no_floating_ip, causing an error when trying to index an empty array for a string that the resource declaration requires.

Error: Invalid index

on ../../../modules/kubespray/modules/compute/main.tf line 670, in resource "openstack_compute_floatingip_associate_v2" "k8s_node": 670: floating_ip = "${var.k8s_node_fips[count.index]}" |---------------- | count.index is 0 | var.k8s_node_fips is empty list of dynamic

According to the error, it seems this may be a interpolation error at ${var.k8s_node_fips[count.index]}. What it seems like is happening is that the variable is empty, but the count is set to a non-zero number, thereby causing TF to error out.

Looking at the original project code, seems like the count variable depends on var.node_root_volume_size_in_gb to be 0 to be set to 0; otherwise will set the count to the value of var.number_of_k8s_nodes_no_floating_ip.

Snippet that the error seems to be occuring at:

resource "openstack_compute_instance_v2" "k8s_node_no_floating_ip_custom_volume_size" {
  name              = "${var.cluster_name}-k8s-node-nf-${count.index+1}"
  count             = "${var.node_root_volume_size_in_gb > 0 ? var.number_of_k8s_nodes_no_floating_ip : 0}"
  availability_zone = "${element(var.az_list, count.index)}"
  image_name        = "${var.image}"
  flavor_id         = "${var.flavor_k8s_node}"
  key_pair          = "${openstack_compute_keypair_v2.k8s.name}"

  block_device {
    uuid                  = "${data.openstack_images_image_v2.vm_image.id}"
    source_type           = "image"
    volume_size           = "${var.node_root_volume_size_in_gb}"
    boot_index            = 0
    destination_type      = "volume"
    delete_on_termination = true
  }

Hope this helps