Trigger lambda when object with specific prefix is created

Question

I am creating a lambda that should be called when objects are created in a certain bucket. I only want it to be called for certain key prefixes. I have seen examples of setting this up using a NotificationConfiguration on the bucket when the bucket is being defined:

NotificationConfiguration:
  LambdaConfigurations:
    - Function: !Ref SomeLambdaArn
      Event: "s3:ObjectCreated:*"
      Filter:
        S3Key:
          Rules:
            - Name: prefix
              Value: zip

However I would rather not modify the bucket's CloudFormation definition. With SAM, lambda event triggers can be defined on the lambda:

  Events:
    BucketEvent1:
      Type: S3
      Properties:
        Bucket: Ref: Bucket1
        Events:
          - 's3:ObjectCreated:*'

But is there a way to restrict it to trigger only for certain prefixes? Alternatively, is there a way to add a NotificationConfiguration to an existing bucket?

So it looks like this would have worked, but it fails because the bucket was defined in a different template. Digging deeper, it seems that I can't configure two lambdas to be notified of the same object creation event due to overlap limitations. I guess I'll have to look into some kind of s3 fanout. — z0r

Jonny Rimek Jonny Rimek · Accepted Answer · 2018-12-29T15:48:31

If I understand you correctly you want to define the prefix in the lambda not the bucket.

This is how I did it, works like a charm as far as I can tell

Resources:
  ConvertToParquet:
  Type: AWS::Serverless::Function 
  Properties:
  CodeUri: handler/convert_to_parquet
  MemorySize: 3008
  Timeout: 90
  Policies: 
    - AmazonS3FullAccess
  Events:
    SummaryCSVCreated:
      Type: S3
      Properties:
        Bucket: !Ref UploadBucket
        Events: s3:ObjectCreated:*
        Filter: 
          S3Key:
            Rules:
              - Name: suffix
                Value: '.txt'

  UploadBucket:
    Type: AWS::S3::Bucket

Trigger lambda when object with specific prefix is created

1 Answers