How Do I Specify a Security Group for Elastic Beanstalk Launch Configuration in CloudFormation Template?

Question

I have the following security group defined in my CloudFormation template:

"APIInstanceSG": {
  "Type": "AWS::EC2::SecurityGroup",
  "Properties": {
    "GroupDescription": "Security Group for Application EC2 Instances,
    "VpcId": "vpc-10a75377",
    "Tags": [{
      "Key": "Name",
      "Value": "APIInstanceSG" }
    }]
  }
}

I also have an Elastic Beanstalk environment defined containing the following inside OptionSettings:

{
  "Namespace": "aws:autoscaling:launchconfiguration",
  "OptionName": "SecurityGroups",
  "Value": { "Ref": "APIInstanceSG" }
}

When I create a stack using this template, the security group is created before CloudFormation attempts to create the EB environment but when it tries to create the EB environment, it failes with the following error:

Configuration validation exception: Invalid option value: 'sg-994fcbe4' (Namespace: 'aws:autoscaling:launchconfiguration', OptionName: 'SecurityGroups'): The security group 'sg-994fcbe4' does not exist

sg-994fcbe4 is the ID of the security group that was created

The Elastic Beanstalk Environment config is as follows:

"AspectAPIEnv": {
  "Type": "AWS::ElasticBeanstalk::Environment",
  "Properties": {
    "ApplicationName": "application-name",
    "EnvironmentName": "environment-name",
    "SolutionStackName": "64bit Amazon Linux 2016.09 v3.1.0 running Node.js",
    "Tier": {
      "Name": "WebServer",
      "Type": "Standard"
    },
    "OptionSettings": [
      {
        "Namespace": "aws:autoscaling:launchconfiguration",
        "OptionName": "EC2KeyName",
        "Value": "ec2-key"
      },
      {
        "Namespace": "aws:autoscaling:launchconfiguration",
        "OptionName": "IamInstanceProfile",
        "Value": "aws-elasticbeanstalk-ec2-role"
      },
      {
        "Namespace": "aws:autoscaling:launchconfiguration",
        "OptionName": "ImageId",
        "Value": "ami-d8356acf"
      },
      {
        "Namespace": "aws:autoscaling:launchconfiguration",
        "OptionName": "InstanceType",
        "Value": "t2.micro"
      },
      {
        "Namespace": "aws:autoscaling:launchconfiguration",
        "OptionName": "SecurityGroups",
        "Value": { "Ref": "APIInstanceSG" }
      },
      {
        "Namespace": "aws:autoscaling:trigger",
        "OptionName": "UpperThreshold",
        "Value": "6000000"
      },
      {
        "Namespace": "aws:autoscaling:updatepolicy:rollingupdate",
        "OptionName": "MaxBatchSize",
        "Value": "1"
      },
      {
        "Namespace": "aws:autoscaling:updatepolicy:rollingupdate",
        "OptionName": "MinInstancesInService",
        "Value": "1"
      },
      {
        "Namespace": "aws:autoscaling:updatepolicy:rollingupdate",
        "OptionName": "RollingUpdateEnabled",
        "Value": "true"
      },
      {
        "Namespace": "aws:autoscaling:updatepolicy:rollingupdate",
        "OptionName": "RollingUpdateType",
        "Value": "Health"
      },
      {
        "Namespace": "aws:elasticbeanstalk:command",
        "OptionName": "BatchSize",
        "Value": "30"
      },
      {
        "Namespace": "aws:elasticbeanstalk:container:nodejs",
        "OptionName": "NodeVersion",
        "Value": "6.2.2"
      },
      {
        "Namespace": "aws:elasticbeanstalk:environment",
        "OptionName": "ServiceRole",
        "Value": "aws-elasticbeanstalk-service-role"
      },
      {
        "Namespace": "aws:elasticbeanstalk:healthreporting:system",
        "OptionName": "SystemType",
        "Value": "enhanced"
      },
      {
        "Namespace": "aws:elasticbeanstalk:managedactions",
        "OptionName": "ManagedActionsEnabled",
        "Value": "true"
      },
      {
        "Namespace": "aws:elasticbeanstalk:managedactions",
        "OptionName": "PreferredStartTime",
        "Value": "SUN:09:02"
      },
      {
        "Namespace": "aws:elasticbeanstalk:managedactions:platformupdate",
        "OptionName": "UpdateLevel",
        "Value": "minor"
      },
      {
        "Namespace": "aws:elb:healthcheck",
        "OptionName": "Interval",
        "Value": "10"
      },
      {
        "Namespace": "aws:elb:loadbalancer",
        "OptionName": "CrossZone",
        "Value": "true"
      },
      {
        "Namespace": "aws:elb:loadbalancer",
        "OptionName": "LoadBalancerHTTPPort",
        "Value": "80"
      },
      {
        "Namespace": "aws:elb:loadbalancer",
        "OptionName": "SecurityGroups",
        "Value": { "Ref": "APILoadBalancerSG" }
      },
      {
        "Namespace": "aws:elb:loadbalancer",
        "OptionName": "ManagedSecurityGroup",
        "Value": { "Ref": "APILoadBalancerSG" }
      },
      {
        "Namespace": "aws:elb:policies",
        "OptionName": "ConnectionDrainingEnabled",
        "Value": "true"
      }
    ],
    "Tags": [
      {
        "Key": "Name",
        "Value": "AspectAPIEnv"
      }
    ]
  },
  "DependsOn": "RDSInstance"
}

Answer 1

After looking at your AWS::ElasticBeanstalk::Environment resource, I was able to reproduce the error you are experiencing. As Marc Young suggested in a comment to your question, you are not specifying a VPC for your environment. Because your security group is in a VPC, it's not accessible from resources that are not also in the same VPC.

To fix it, you must add the following configuration option to your environment:

{
  "Namespace" : "aws:ec2:vpc",
  "OptionName" : "VPCId",
  "Value" : "vpc-10a75377"
},

If you specify a VPC, creating a stack with the updated template will fail with an error message saying that you also need to specify the environment subnets, so you will have to add the following options:

{
  "Namespace" : "aws:ec2:vpc",
  "OptionName" : "Subnets",
  "Value" : <insert the subnet for your instances here>
},
{
  "Namespace" : "aws:ec2:vpc",
  "OptionName" : "ELBSubnets",
  "Value" : <insert the subnet for your load balancer here>
}

You can check a working example of a Beanstalk application in a VPC in the Elastic Beanstalk CloudFormation sample templates.

Answer 2

To overcome this:

You need to change the EB Security Group from AWS CLI, you cannot do it from AWS Web Console.

Considering you have already AWS CLI installed, you will need to do this command if you want to change the Security Group:

aws elasticbeanstalk update-environment –environment-name –option-settings Namespace=aws:autoscaling:launchconfiguration,OptionName=SecurityGroups,Value=””

Source

Answer 3

You should be setting the DependsOn attribute in your LC definition to ensure it exists before the SG during stack creation. Otherwise you can't guarantee the reference will work.

"APIInstanceSG": {
  "Type": "AWS::EC2::SecurityGroup",
  "Properties": {
    "GroupDescription": "Security Group for Application EC2 Instances,
    "VpcId": "vpc-10a75377",
    "Tags": [{
      "Key": "Name",
      "Value": "APIInstanceSG" }
    }]
  },
  "DependsOn" : "APIInstanceSG"
}

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-dependson.html

Answer 4

In your template, instead of

"DependsOn" : "RDSInstance"

write:

"DependsOn": ["APIInstanceSG", "RDSInstance"]

More info: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-dependson.html