How to Keep Usage of Terraform aws_security_group DRY

Question

I've written a simple module to provision a variable AZ numbered AWS VPC. It creates the route tables, gateways, routes, etc., but I'm having trouble keeping the security groups part DRY, i.e. keeping the module re-usable when specifying security groups.

This is as close as I can get:

varibles.tf:

variable "staging_security_groups" {
  type = "list"
  default = [ {
      "name" = "staging_ssh"
      "from port" = "22"
      "to port" = "22"
      "protocol" = "tcp"
      "cidrs" = "10.0.0.5/32,10.0.0.50/32,10.0.0.200/32"
      "description" = "Port 22"
  } ]
}

main.tf:

resource "aws_security_group" "this_security_group" {
  count = "${length(var.security_groups)}"

  name        = "${lookup(var.security_groups[count.index], "name")}"
  description = "${lookup(var.security_groups[count.index], "description")}"
  vpc_id      = "${aws_vpc.this_vpc.id}"

  ingress {
    from_port   = "${lookup(var.security_groups[count.index], "from port")}"
    to_port     = "${lookup(var.security_groups[count.index], "to port")}"
    protocol    = "${lookup(var.security_groups[count.index], "protocol")}"
    cidr_blocks = ["${split(",", lookup(var.security_groups[count.index], "cidrs"))}"]
  }

  egress {
    from_port       = 0
    to_port         = 0
    protocol        = "-1"
    cidr_blocks     = ["0.0.0.0/0"]
  }

  tags {
    Name = "${lookup(var.security_groups[count.index], "name")}"
    environment = "${var.name}"
    terraform = "true"
  }
}

Now this is fine, as long as what you want is to create a security group per port :) What I really need, is some way to call ingress the number of times that there are values in the variable staging_security_groups[THE SECURITY GROUP].from_port (please excuse the made-up notation).

Answer 1

You could look at using aws_security_group_rule instead of having your rules inline. You can then create a module like this:

module/sg/sg.tf

resource "aws_security_group" "default" {
  name        = "${var.security_group_name}"
  description = "${var.security_group_name} group managed by Terraform"

  vpc_id = "${var.vpc_id}"
}

resource "aws_security_group_rule" "egress" {
  type              = "egress"
  from_port         = 0
  to_port           = 0
  protocol          = "-1"
  cidr_blocks       = ["0.0.0.0/0"]
  description       = "All egress traffic"
  security_group_id = "${aws_security_group.default.id}"
}

resource "aws_security_group_rule" "tcp" {
  count             = "${var.tcp_ports == "default_null" ? 0 : length(split(",", var.tcp_ports))}"
  type              = "ingress"
  from_port         = "${element(split(",", var.tcp_ports), count.index)}"
  to_port           = "${element(split(",", var.tcp_ports), count.index)}"
  protocol          = "tcp"
  cidr_blocks       = ["${var.cidrs}"]
  description       = ""
  security_group_id = "${aws_security_group.default.id}"
}

resource "aws_security_group_rule" "udp" {
  count             = "${var.udp_ports == "default_null" ? 0 : length(split(",", var.udp_ports))}"
  type              = "ingress"
  from_port         = "${element(split(",", var.udp_ports), count.index)}"
  to_port           = "${element(split(",", var.udp_ports), count.index)}"
  protocol          = "udp"
  cidr_blocks       = ["${var.cidrs}"]
  description       = ""
  security_group_id = "${aws_security_group.default.id}"
}

modules/sg/variables.tf

variable "tcp_ports" {
  default = "default_null"
}

variable "udp_ports" {
  default = "default_null"
}

variable "cidrs" {
  type = "list"
}

variable "security_group_name" {}

variable "vpc_id" {}

Use the module in your main.tf

module "sg1" {
  source              = "modules/sg"
  tcp_ports           = "22,80,443"
  cidrs               = ["10.0.0.5/32", "10.0.0.50/32", "10.0.0.200/32"]
  security_group_name = "SomeGroup"
  vpc_id              = "${aws_vpc.this_vpc.id}"
}

module "sg2" {
  source              = "modules/sg"
  tcp_ports           = "22,80,443"
  cidrs               = ["10.0.0.5/32", "10.0.0.50/32", "10.0.0.200/32"]
  security_group_name = "SomeOtherGroup"
  vpc_id              = "${aws_vpc.this_vpc.id}"
}

References:

For why optionally excluding a resource with count looks like this (source):

count             = "${var.udp_ports == "default_null" ? 0 : length(split(",", var.udp_ports))}"

And the variable is set to:

variable "udp_ports" {
  default = "default_null"
}

Answer 2

I managed to create really simple yet dynamic security group module that you can use. Idea here is to have ability to add any port you desire, and add to that port any range of ips you like. You can even remove egress from module as it will be created by default, or follow idea i used in ingress so you have granular egress rules (if you wish so).

module/sg/sg.tf

  data "aws_subnet_ids" "selected" {
  vpc_id = "${var.data_vpc_id}"
}

resource "aws_security_group" "main" {
  name        = "${var.sg_name}-sg"
  vpc_id      = "${var.data_vpc_id}"
  description = "Managed by Terraform"
  ingress     = ["${var.ingress}"]

  lifecycle {
    create_before_destroy = true
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

module/sg/vars.tf

variable "sg_name" {}

variable "data_vpc_id" {}

variable "ingress" {
  type    = "list"
  default = []
}

ingress var needs to be type list. If you call vpc id manually you dont need data bit in module, im calling my vpc_id from terraform state that is stored in s3.

main.tf

module "aws_security_group" {
  source = "module/sg/"

  sg_name     = "name_of_sg"
  data_vpc_id = "${data.terraform_remote_state.vpc.vpc_id}"

  ingress = [
    {
      from_port   = 22
      to_port     = 22
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
      description = "Managed by Terraform"
    },
    {
      from_port   = 0
      to_port     = 100
      protocol    = "tcp"
      cidr_blocks = ["10.10.10.10/32"]
      description = "Managed by Terraform"
    },
    {
      from_port   = 2222
      to_port     = 2222
      protocol    = "tcp"
      cidr_blocks = ["100.100.100.0/24"]
      description = "Managed by Terraform"
    },
  ]
}

You can add as many ingress blocks you like, i have only 3 for test purposes. Hope this helps.
Note: You can follow this idea for many resources like RDS, where you need to specify parameters in parameter group or even tags. Cheers

Answer 3

Not sure if it was available at the time Brandon Miller's answer was written, but avoid count loops as they are ordered. So if you add or delete one port, it will cause all rules after it to be rebuilt as they rely on the count index, which changes. Far better to use a for_each loop. Make sure you use set not lists for this eg

variable "tcp_ports" {
  default = [ ]
  # or maybe default = [ "22", "443" ]
  type = set(string)
}

resource "aws_security_group_rule" "tcp" {
  for_each                 = var.tcp_ports
  description              = "Allow ${var.cdir} to connect to TCP port ${each.key}"
  type                     = "ingress"
  from_port                = each.key
  to_port                  = each.key
  protocol                 = "tcp"
  cidr_blocks              = var.cdir
  security_group_id = aws_security_group.default.id
}

Now you can add and delete ports without incurring unnecessary create and destroys

you you cant alter your data from lists to sets for any reason just wrap it eg

toset(var.tcp_ports)

or use a local to munge your data accordingly. You can also use maps as well