Cloudformation - how to set filter policy of SNS subscription in code?

Question

UPDATE: Cloudformation now supports SNS Topic Filters, so this question is not relevant anymore, no custom plugins or code is needed.

I am building a system with a number of SNS topics, and a number of Lambdas which are each reading messages from their assigned SQS queue. The SQS queues are subscribed to the SNS topics, but also have a filter policy so the messages will end up in the relevant SQS queues.

It works well when I set up the subscriptions in the AWS console.

Now I'm trying to do the same in my code, but the AWS Cloudformation documentation does not describe how to add a filter policy to a subscription. Based on the python examples here, I tried the following:

  StopOperationSubscription:
    Type: "AWS::SNS::Subscription"
    Properties:
      Protocol: sqs
      TopicArn: 
        Ref: StatusTopic
      Endpoint: 
        Fn::GetAtt: [StopActionQueue, Arn]
      FilterPolicy: '{"value": ["stop"]}'

But then I get this error:

An error occurred: StopOperationSubscription - Encountered unsupported property FilterPolicy.

How can I set the filter policy that I need, using CloudFormation? And If that's not supported, what do you suggest as an alternative?

I want it to be set up automatically when I deploy my serverless app, with no manual steps required.

Answer 1

Cloudformation just started to support FilterPolicy yesterday. I have been struggling for a while too :)

Syntax

JSON

{
  "Type" : "AWS::SNS::Subscription",
  "Properties" : {
    "DeliveryPolicy" : JSON object,
    "Endpoint" : String,
    "FilterPolicy" : JSON object,
    "Protocol" : String,
    "RawMessageDelivery" : Boolean,
    "Region" : String,
    "TopicArn" : String
  }
}

YAML

Type: "AWS::SNS::Subscription"
Properties:
  DeliveryPolicy: JSON object
  Endpoint: String
  FilterPolicy: JSON object
  Protocol: String
  RawMessageDelivery: Boolean,
  Region: String
  TopicArn: String

Ref:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sns-subscription.html#cfn-sns-subscription-filterpolicy

https://aws.amazon.com/blogs/compute/managing-amazon-sns-subscription-attributes-with-aws-cloudformation/

Answer 2

I fixed it like this:

serverless.yml

plugins:
  - serverless-plugin-scripts
custom:
  scripts:
    commands:
      update-topic-filters: sls invoke local -f configureSubscriptions --path resources/lambdaTopicFilters.json
    hooks:
      before:deploy:finalize: sls update-topic-filters


functions:
  configureSubscriptions:
    handler: src/configurationLambdas/configureSubscriptions.main
    # Only invoked when deploying - therefore, no permissions or triggers are needed.

configureSubscriptions.js

import AWS from 'aws-sdk'

const nameFromArn = arn => arn.split(':').pop()
const lambdaNameFromArn = arn =>
  nameFromArn(arn)
    .split('-')
    .pop()

exports.main = async event => {
  const sns = new AWS.SNS({ apiVersion: '2010-03-31' })
  const params = {}
  const { Topics } = await sns.listTopics(params).promise()
  for (const { TopicArn } of Topics) {
    const topicName = nameFromArn(TopicArn)
    const filtersForTopic = event[topicName]
    if (!filtersForTopic) {
      continue
    }
    const { Subscriptions } = await sns.listSubscriptionsByTopic({ TopicArn }).promise()
    for (const { Protocol, Endpoint, SubscriptionArn } of Subscriptions) {
      if (Protocol === 'lambda') {
        const lambdaName = lambdaNameFromArn(Endpoint)
        const filterForLambda = filtersForTopic[lambdaName]
        if (!filterForLambda) {
          continue
        }

        const setPolicyParams = {
          AttributeName: 'FilterPolicy',
          SubscriptionArn,
          AttributeValue: JSON.stringify(filterForLambda),
        }
        await sns.setSubscriptionAttributes(setPolicyParams).promise()
        // eslint-disable-next-line no-console
        console.log('Subscription filters has been set')
      }
    }
  }
}

Top level is the different topic names, next level is the lambda names, and the third level is the filter policies for the related subscriptions:

lambdaTopicFilters.json

{
  "user-event": {
    "activateUser": {
      "valueType": ["status"],
      "value": ["awaiting_activation"]
    },
    "findActivities": {
      "messageType": ["event"],
      "value": ["awaiting_activity_data"],
      "valueType": ["status"]
    }
  },
  "system-event": {
    "startStopProcess": {
      "valueType": ["status"],
      "value": ["activated", "aborted", "limit_reached"]
    }
  }
}

Answer 3

If you are using serverless it is now supporting sns filter natively

functions:
  pets:
    handler: pets.handler
    events:
      - sns:
          topicName: pets
          filterPolicy:
            pet:
              - dog
              - cat

https://serverless.com/framework/docs/providers/aws/events/sns#setting-a-filter-policy