Unresolved Parameter Store SecureString dynamic reference in CloudFormation template

Question

According to AWS, when deploying infrastructure that requires secrets, i.e. passwords or similar, with CloudFormation, one popular solution consists in using the Parameter Store's SecureStrings from SSM.

However, despite existing CFN documentation describing step-by-step how to use the Dynamic References within the CFN templates, I can not manage to make use of the actual value of the SecureStrings.

Assuming the following JSON representation of the an existing SecureString stored in the SSM Parameter Store:

{
  "MyRedshiftMasterUserPassword": {
    "value": "Abcd2019",
    "type": "SecureString"
  }
}

and a YAML CFN template that uses it as stated in the documentation:

Resources
  Redshift:
    Type: 'AWS::Redshift::Cluster'
    Properties:
      NodeType: dc2.large
      NumberOfNodes: !Ref RedshiftNodes
      ClusterType: multi-node
      AutomatedSnapshotRetentionPeriod: !Ref AutomatedSnapshotRetentionPeriod
      DBName: datawarehouse_v1
      MasterUsername: !Ref RedshiftMasterUsername
      MasterUserPassword: '{{resolve:ssm-secure:MyRedshiftMasterUserPassword:1}}'

The above solution does not to work so either I am defining the template incorrectly or the support for this feature is not properly implemented which seems odd to me considering that it comes from AWS.

Particularly, I came across the following errors that all end up as UPDATE_FAILED stack:

Whenever the referenced Parameter Name to be resolved is long enough CloudFormation complains:

The parameter MasterUserPassword is not a valid password because it is longer than 64 characters. (Service: AmazonRedshift; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 7be9bd43-2927-11e9-aa88-29bbdcae859e)

Additionally, even though specifically mentioned that slashes can be used in the template references, e.g. /infrastructure/datawarehouse/redshift/MyRedshiftMasterUserPassword following error is issued:

The parameter MasterUserPassword is not a valid password. Only printable ASCII characters except for '/', '@', '"', ' ', '\', ''' may be used. (Service: AmazonRedshift; Status Code: 400; Error Code: InvalidParameterValue)

thus, as result referenced SecureString do not seem to be compatible with the SSM ParameterStore hierarchy (parameters with slashes).

Moreover, removing any of the previously reported invalid character from the parameter name, then it complains of the following:

The parameter MasterUserPassword must contain at least 1 upper case letter. (Service: AmazonRedshift; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 90a263bd-2929-11e9-80c0-ffcecf297c44)

In the end, although using a basic short non-slashed Parameter name in a template allows the stack to finish the Update operation dynamic reference still does not occur as the actual value used turns out to be supplied Parameter Name rather than the value referenced by this, e.g MyRedshiftMasterUserPassword instead of Abcd2019.

I am aware that AWS Secrets Manager could also be used but it is not free.

How did you verify that the actual value turned out to be parameter name rather than the value? Also, the errors you specified is not for the parameter name but rather the value and 'Abcd2019' does seem correct. I have successfully used securestring just like the way you presented so it does seem correct. Are you trying to retrieve it using some SDK like boto3? — Biplob Biswas
I verified the final value used by trying connecting to the Redshift cluster, as it only authenticates with the name, not the value as a password. Effectively, the SecureString 'Abcd2019' is a valid password, but since it ends up using the name, not the value, then it fails to meet the constraints therefore errors. — blamblam
Can you verify the values of these 2 aws commands: aws ssm get-parameter --name MyRedshiftMasterUserPassword and aws ssm get-parameter --name MyRedshiftMasterUserPassword --with-decryption? Because otherwise it looks good to me. — Biplob Biswas
Just checked the returning values from executing the awscli commands and result is successfully resolved. — blamblam

blamblam blamblam · Accepted Answer · 2019-02-12T15:44:27

Opened support case with AWS requesting guidance for this particular strange behaviour of CloudFormation.

According to the support team, effectively this is indeed a known bug for the CloudFormation service without estimated time to fix though. The resolution of the a SSM Parameter Store SecureString parameter when used as a Dynamic Reference in the particular case of the RedshiftMasterUserPassword property despite referenced in the documentation does not get properly resolved and parameter name is used instead.

Alternatively, they offer 2 workarounds whilst the issue gets fixed:

Get the 'MasterUserPassword' for Redshift from input parameter with property NoEcho set to true. NoEcho property allows you to mask the password value and you don't need to store the password in a template file. However, each time you update the stack you need to enter the password as input parameter. For your reference, below code snippet will be useful.

Second option, which is more versatile:

Define a Lambda backed Custom Resource in your template file, which queries the SSM service and returns the password to CloudFormation. In this scenario, You need to write a Custom Code for lambda function which uses AWS GetParameter API call to retrieve the value of SSM Secure String Parameter and returns the decrypted value to CloudFormation.

Other supported properties for dynamic reference seem to work fine.

Unresolved Parameter Store SecureString dynamic reference in CloudFormation template

1 Answers