RDS issues with custom VPC Subnets

Question

I am trying to use CDK to define a Serverless Postgres Aurora cluster but keep running into issues with regards to the VPC subnets either being "invalid" or "not existing", depending on which db cluster construct I attempt to use. In my setup, I have 2 Stacks: 1 for the VPC, and 1 for the RDS.

This is the contents of my Vpc Stack:

const vpc = new Vpc(this, 'Vpc');

const privateSubnetIds = vpc.selectSubnets({
    subnetType: SubnetType.PRIVATE
}).subnetIds;

const rdsSecurityGroup = new SecurityGroup(this, 'RdsSecurityGroup', {
    securityGroupName: 'rds-security-group',
    allowAllOutbound: true,
    description: `RDS cluster security group`,
    vpc: vpc
});

...

// The rest of the file defines exports.

Case 1:

Initially, I tried using the CfnDBCluster as the DatabaseCluster does not allow you to directly define engineMode: 'serverless' and enableHttpEndpoint: true. Below is the contents of the RDS Stack using the CfnDBCluster construct:

// The beginning of the file imports all the VPC exports from the VPC Stack:
//   subnetIds (for the private subnet), securityGroupId
...

const databaseSecret = new DatabaseSecret(this, 'secret', {
    username: 'admin'
});

const secretArn = databaseSecret.secretArn;

const dbSubnetGroup = new CfnDBSubnetGroup(this, "DbSubnetGroup", {
    dbSubnetGroupDescription: `Database cluster subnet group`,
    subnetIds: subnetIds
});

const dbCluster = new CfnDBCluster(this, 'DbCluster', {
    dbClusterIdentifier: 'aurora-cluster',
    engine: 'aurora-postgresql',
    engineMode: 'serverless',
    databaseName: DB_NAME,
    masterUsername: databaseSecret.secretValueFromJson('username').toString(),
    masterUserPassword: databaseSecret.secretValueFromJson('password').toString(),
    enableHttpEndpoint: true,
    scalingConfiguration: {
        autoPause: true,
        minCapacity: 1,
        maxCapacity: 16,
        secondsUntilAutoPause: 300
    },
    vpcSecurityGroupIds: [securityGroupId],
    dbSubnetGroupName: dbSubnetGroup.dbSubnetGroupName
});

Using the CfnDBCluster construct, I get the following error:

Some input subnets in :[subnet-044631b3e615d752c,subnet-05c2881d9b13195ef,subnet-03c63ec89ae49a748] are invalid. (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 5c4e6237-6527-46a6-9ed4-1bc46c38dce0)

I am able to verify that those Subnets do exist before the RDS Stack is run.

Case 2:

After failing to get the CfnDBCluster example above working, I tried using the DatabaseCluster construct with raw overrides. Below is the contents of the RDS Stack using the DatabaseCluster construct:

// The beginning of the file imports all the VPC exports from the VPC Stack:
//   subnetIds (for the private subnet), securityGroupId, vpcId, AZs, vpc (using Vpc.fromAttributes)
...

const dbCluster = new DatabaseCluster(this, 'DbCluster', {
    engine: DatabaseClusterEngine.auroraPostgres({
        version: AuroraPostgresEngineVersion.VER_10_7
    }),
    masterUser: {
        username: databaseSecret.secretValueFromJson('username').toString(),
        password: databaseSecret.secretValueFromJson('password')
    },
    instanceProps: {
        vpc: vpc,
        vpcSubnets: {
            subnetType: SubnetType.PRIVATE
        }
    },
});

const cfnDbCluster = dbCluster.node.defaultChild as CfnDBCluster;
cfnDbCluster.addPropertyOverride('DbClusterIdentifier', 'rds-cluster');
cfnDbCluster.addPropertyOverride('EngineMode', 'serverless');
cfnDbCluster.addPropertyOverride('DatabaseName', DB_NAME);
cfnDbCluster.addPropertyOverride('EnableHttpEndpoint', true);
cfnDbCluster.addPropertyOverride('ScalingConfiguration.AutoPause', true);
cfnDbCluster.addPropertyOverride('ScalingConfiguration.MinCapacity', 1);
cfnDbCluster.addPropertyOverride('ScalingConfiguration.MaxCapacity', 16);
cfnDbCluster.addPropertyOverride('ScalingConfiguration.SecondsUntilAutoPause', 300);
cfnDbCluster.addPropertyOverride('VpcSecurityGroupIds', subnetIds);

Using the DatabaseCluster construct, I get the following error:

There are no 'Private' subnet groups in this VPC. Available types:

I am able to verify that the VPC does have a Private subnet, I also verified that it was properly imported and that the Subnets all have the expected tags i.e. key: 'aws-cdk:subnet-type' value: 'Private'

This issue has me blocked and confused, I cannot figure out why either of these issues are manifesting and would appreciate any guidance offered on helping resolve this issue.

References:

Notes:

I am using CDK version 1.56.0 with Typescript

If you manually create the aurora severless with the same subnets and parameters using AWS console, does it all work? If it works, it means that only your CDK code needs correction. If if does not, it means that your architecture needs re-checking. — Marcin
@Marcin yes, manually creating the aurora cluster with the same subnets and parameters using the console does work. — Alestin Sphere
are you aware of this issue in github github.com/aws/aws-cdk/issues/929? If I understand the issue correctly you can't create a serverless aurora DB right now. you can track the status here github.com/aws/aws-cdk/projects/5 — Jonny Rimek

Amit Baranes Amit Baranes · Accepted Answer · 2020-09-08T20:26:37

In case you visiting this page after getting-

Some input subnets in :[subnet-XXXX,subnet-YYYY,subnet-ZZZZ] are invalid.

You probably checked and confirmed that these subnets do not exist and knock your head struggling to find where the hell these subnets are coming from.

The reason CDK still point to these subnets is since cdk.context.json is still contains values from last deployments.

From the docs-

Context values are key-value pairs that can be associated with a stack or construct. The AWS CDK uses context to cache information from your AWS account, such as the Availability Zones in your account or the Amazon Machine Image (AMI) IDs used to start your instances.

Replace all the JSON content to a valid one ( {} ) and re-deploy the stack.

RDS issues with custom VPC Subnets

1 Answers