Create an SNS message from an existing buckets S3 create event using cloudformation/serverless

Question

I have an existing s3 bucket not created by the serverless/cloudformation stack. I also have a serverless stack with a bunch of lambdas that are triggered off of s3 create events with different prefixes. I have a need to send some s3 create events over SNS so that it can be used in another system.

In my serverless.yaml I have a resources section where I can create an SNS Topic and Policy. This appears to work fine:

resources:
  Resources:
    # Create a topic for the created s3 documents to be published on.
    MyS3ToSnsTopic:
      Type: AWS::SNS::Topic
      Properties:
        DisplayName: 'Topic to notify new s3-to-sns s3 document was created'
        TopicName: s3-to-sns
    # Policy for Bucket to publish sns messages to `s3-to-sns` topic
    MyS3ToSnsTopicPolicy:
      Type: AWS::SNS::TopicPolicy
      Properties:
        PolicyDocument:
          Id: MyS3ToSnsTopicPolicy
          Version: '2012-10-17'
          Statement:
          - Sid: s3-to-sns-id
            Effect: Allow
            Principal:
              AWS: "*"
            Action: sns:Publish
            Resource: !Join [':', ['arn:aws:sns:us-west-2', Ref: 'AWS::AccountId', 's3-to-sns']]
            Condition:
              ArnLike:
                "aws:SourceArn": "arn:aws:s3:*:*:${self:custom.bucketName.${self:provider.stage}}"
        Topics:
        - !Ref MyS3ToSnsTopic

When I go to the bucket -> properties -> events I can manually add an event that listens to a prefix/suffix, and can publish to that topic. So the last piece is to create the event with cloudformation instead of manually doing it.

When I have serverless create a lambda trigger based on the create event it generates the following:

SomeFunctionNameCustomS31:
  Type: Custom::S3
  Version: 1
  DependsOn:
  - SomeFunctionNameLambdaFunction
  - CustomDashresourceDashexistingDashs3LambdaFunction
  Properties:
    ServiceToken:
      Fn::GetAtt:
      - CustomDashresourceDashexistingDashs3LambdaFunction
      - Arn
    FunctionName: some-function-name
    BucketName: some-bucket-name
    BucketConfigs:
    - Event: s3:ObjectCreated:*
      Rules:
      - Prefix: metrics/
      - Suffix: ".json"

Is there a way to do something similar for SNS, but instead of using a FunctionName use something like TopicName? What other attributes would need to be added or changed? I've seen recommendations about NotificationConfiguration when you create the bucket with the stack, but my bucket wasn't created with my stack.

Have you seen docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/…? — Oleksii Donoha
@OleksiiDonoha I mentioned seeing the NotificationConfiguration at the end, but I'm not sure how to take advantage of that when the bucket was manually created earlier. — openam
I see 2 ways then: a) import bucket into stack, semi-manual (aws.amazon.com/blogs/aws/…); b) write custom resource lambda (similar to what you have in your question) which takes topic name as an option and utilizes SDK to subscribe topic to events — Oleksii Donoha
There isn't any way to make a custom s3 event resource to send sns messages? I don't really want to run a lambda to just publish a message. — openam
That's what custom resource is. It's a lambda which does the work. I'm not familiar with serverless framework, maybe someone has better insight. What I said applies to Cloudformation in general — Oleksii Donoha

noetix noetix · Accepted Answer · 2020-05-20T22:54:32

The framework allows you to customise resources it creates, and you can always configure any resources you manually create through the Resources block. Both allowing you to set NotificationConfiguration which allows for SNS subscription.

resources:
  Resources:
    MyBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: ${self:custom.bucketName}
        NotificationConfiguration:
          TopicConfigurations:
            - Event: s3:ObjectCreated:*
              Topic:
                Ref: MySnsTopic

The documentation also details how you would customize a framework created S3 bucket.

Create an SNS message from an existing buckets S3 create event using cloudformation/serverless

1 Answers