Unable to add GSI to DynamoDB table using CloudFormation

Question

I have an existing DynamoDB table that is defined as part of a CloudFormation stack. According the the CFN AWS::DynamoDB::Table documentation the GlobalSecondaryIndexes attribute does not require replacement. It even goes into details with the following caveats.

You can delete or add one global secondary index without interruption.

As well as the following...

If you update a table to include a new global secondary index, AWS CloudFormation initiates the index creation and then proceeds with the stack update. AWS CloudFormation doesn't wait for the index to complete creation because the backfilling phase can take a long time, depending on the size of the table.

However, in practice when I attempt to perform an update I get the following error message:

CloudFormation cannot update a stack when a custom-named resource requires replacing. Rename mytablename and update the stack again.

Since I'm adding a GSI that uses a new attribute I'm forced to modify AttributeDefinitions which says it does require replacement. However, even when I try to add a GSI with only existing attributes defined in the AttributeDefinitions I still get the same error message.

Here is the snippet from my original CFN definition for my table:

{
  "myTable": {
    "Type": "AWS::DynamoDB::Table",
    "Properties": {
      "TableName": "mytablename",
      "AttributeDefinitions": [
        {
          "AttributeName": "entryId",
          "AttributeType": "S"
        },
        {
          "AttributeName": "entryName",
          "AttributeType": "S"
        },
        {
          "AttributeName": "appId",
          "AttributeType": "S"
        }
      ],
      "KeySchema": [
        {
          "KeyType": "HASH",
          "AttributeName": "entryId"
        },
        {
          "KeyType": "RANGE",
          "AttributeName": "entryName"
        }
      ],
      "ProvisionedThroughput": {
        "ReadCapacityUnits": {
          "Ref": "readThroughput"
        },
        "WriteCapacityUnits": {
          "Ref": "writeThroughput"
        }
      },
      "GlobalSecondaryIndexes": [
        {
            "IndexName": "appId-index",
          "KeySchema": [
            {
              "KeyType": "HASH",
              "AttributeName": "appId"
            }
          ],
          "Projection": {
            "ProjectionType": "KEYS_ONLY"
          },
          "ProvisionedThroughput": {
            "ReadCapacityUnits": {
              "Ref": "readThroughput"
            },
            "WriteCapacityUnits": {
              "Ref": "writeThroughput"
            }
          }
        }
      ]
    }
  }
}

Here is what I want to update it to:

{
  "myTable": {
    "Type": "AWS::DynamoDB::Table",
    "Properties": {
      "TableName": "mytablename",
      "AttributeDefinitions": [
        {
          "AttributeName": "entryId",
          "AttributeType": "S"
        },
        {
          "AttributeName": "entryName",
          "AttributeType": "S"
        },
        {
          "AttributeName": "appId",
          "AttributeType": "S"
        },
        {
          "AttributeName": "userId",
          "AttributeType": "S"
        }
      ],
      "KeySchema": [
        {
          "KeyType": "HASH",
          "AttributeName": "entryId"
        },
        {
          "KeyType": "RANGE",
          "AttributeName": "entryName"
        }
      ],
      "ProvisionedThroughput": {
        "ReadCapacityUnits": {
          "Ref": "readThroughput"
        },
        "WriteCapacityUnits": {
          "Ref": "writeThroughput"
        }
      },
      "GlobalSecondaryIndexes": [
        {
            "IndexName": "appId-index",
          "KeySchema": [
            {
              "KeyType": "HASH",
              "AttributeName": "appId"
            }
          ],
          "Projection": {
            "ProjectionType": "KEYS_ONLY"
          },
          "ProvisionedThroughput": {
            "ReadCapacityUnits": {
              "Ref": "readThroughput"
            },
            "WriteCapacityUnits": {
              "Ref": "writeThroughput"
            }
          }
        },
        {
          "IndexName": "userId-index",
          "KeySchema": [
            {
              "KeyType": "HASH",
              "AttributeName": "userId"
            }
          ],
          "Projection": {
            "ProjectionType": "KEYS_ONLY"
          },
          "ProvisionedThroughput": {
            "ReadCapacityUnits": {
              "Ref": "readThroughput"
            },
            "WriteCapacityUnits": {
              "Ref": "writeThroughput"
            }
          }
        }
      ]
    }
  }
}

However, like I mentioned before even if I do not define userId in the AttributeDefinitions and use an existing attribute in a new GSI definition it does not work and fails with the same error message.

I can confirm the same happens when I try to add a GSI without any change to AttributeDefinitions. The documentation also states the following: "Update requires: Updates are not supported. with the following exceptions: If you update only the provisioned throughput values of global secondary indexes, you can update the table without interruption. You can delete or add one global secondary index without interruption. If you do both in the same update (for example, by changing the index's logical ID), the update fails." — Nick Pestov

Todd Feak Todd Feak · Accepted Answer · 2016-04-30T02:07:46

I had the same error today and got an answer from Amazon tech support. The problem is that you supplied a TableName field. CloudFormation wants to be in charge of naming your tables for you. Apparently, when you supply your own name for them, this is the error you get on a update that replaces the table (not sure why it needs to replace, but that's what the doc says)

For me, this makes CloudFormation utterly useless for maintaining my DynamoDB tables. I'd have to build in configuration so that my code could dynamically tell what the random table name was that CloudFormation generated for me.

Unable to add GSI to DynamoDB table using CloudFormation

5 Answers

Workaround A

Workaround B