SNS + CloudFormation

3
votes

I'm using AWS CloudFormation to build a stack for a microservice. My AWS CloudFormation template creates resources like: a Lambda function, an SNS topic and API Gateway.

This microservice does some work and publishes messages to the SNS topic. Other microservices subscribe to this topic.

The problem I'm facing is that when I upgrade my microservice's CloudFormation template (sometimes I need to redeploy it, and recreate all resources), the SNS topic changes its ARN. Hence, all microservices that use this topic need to change as well.

I think I could create a separate CloudFormation template for the SNS topic (I have more than one per microservice).

  • Will this be a good approach?
  • If not, what's the recommended way?
3
microservicesamazon-snsamazon-cloudformation
Can you share your template? Or share what you are changing when this happens? – George M Whitaker
In this case, I am upgradig the serverless framework version. So I know that it will delete and recreate the stack. – p.magalhaes
How do you share the SNS topic between the publisher and subscriber stacks? – rbarni

3 Answers

1
votes

As per AWS CloudFormation docs, there are 3 properties available for an SNS topic, and only a change to the third property (i.e. TopicName) would lead to a replacement of an SNS resource (and leading to a generation of a new ARN).

Since you're nuking the resources in CloudFormation stack and then re-creating them (you mentioned that that's how the serverless framework you're using works), you can preserve the SNS topic ARN by specifying the TopicName attribute for the SNS topic in your template (it's only when one doesn't specify a TopicName is when CloudFormation comes up and assigns a random name, otherwise it uses the name that the user specifies).

That being said, your scenario does seem a little unnatural to me. If you aren't already managing subscriptions to that topic via CloudFormation, I'd recommend doing that, and it should automatically create SNS subscriptions on the replaced SNS topic for you, if at all SNS topic ARN changes.

Ref: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-sns-topic.html

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sns-subscription.html

0
votes

@p.magalhaes - Ideally you should not be deleting the stack and creating it again, rather you should be updating the cloudformation stack. And if you update your stack it wont delete SNS topic as there won't be any change in that. However if you have a specific need of deleting and creating the stack you can create separate stacks for all of your components using nested stack (Reference - https://aws.amazon.com/blogs/devops/use-nested-stacks-to-create-reusable-templates-and-support-role-specialization/ )

0
votes

If you need to dynamically link to your SNS topic's ARN, you should create an environmental variable within your Lambda resource and reference your topic's ARN, like so:

AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31 Resources: MyTopic: Type: AWS::SNS::Topic MyLambdaFunction: Type: AWS::Lambda::Function Properties: Handler: index.handler Runtime: nodejs8.10 Code: ./path/to/index.js Role: [Role ARN here] MemorySize: 128 Timeout: 3 Environment: Variables: SNS_TOPIC: !Ref MyTopic

If multiple stacks need to access your SNS Topic, I suggest creating a separate stack for your SNS topic and then putting the topic ARN in other templates as a parameter, and similarly reference that parameter, like so:

AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31 Parameters: MyTopicArn: Type: String Resources: MyLambdaFunction: Type: AWS::Lambda::Function Properties: Handler: index.handler Runtime: nodejs8.10 Code: ./path/to/index.js Role: [Role ARN here] MemorySize: 128 Timeout: 3 Environment: Variables: SNS_TOPIC: !Ref MyTopicArn

This way, if you have to replace your SNS topic, you can just update the other stacks' parameters when you deploy them next.