Using Serverless Framework how to Logging Amazon S3 Object-Level operation using Cloud Watch events

Question

I have an S3 Resource on which I put and remove objects. Those operations are client-side operations meaning that there is no lambda function involve in those actions. The backend was created using serverless.

Here is the S3 resource under resource key in serverless.yml

resources:
  Resources:
    AttachmentsBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: ${self:provider.environment.PROJECTS_S3_BUCKET}
        CorsConfiguration:
          CorsRules:
            - AllowedOrigins:
                - '*'
              AllowedHeaders:
                - '*'
              AllowedMethods:
                - GET
                - PUT
                - POST
                - DELETE
                - HEAD
              MaxAge: 3000

    BucketPolicy:
      Type: AWS::S3::BucketPolicy
      Properties:
        Bucket: !Ref AttachmentsBucket
        PolicyDocument:
          Id: MyPolicy
          Version: "2012-10-17"
          Statement:
            - Sid: PublicReadWriteBucketPolicy
              Effect: Allow
              Principal: '*'
              Action:
                - 's3:GetObject'
                - 's3:PutObject'
              Resource: 'arn:aws:s3:::${self:provider.environment.PROJECTS_S3_BUCKET}/*'

I need to log Amazon S3 Object-Level operation using Cloudwatch events (PUT and DELETE objects) so I can later use those logs (as a source for a Step function)

Could someone please let me know how to log Amazon S3 Object-Level operation using CloudWatch events in Serverless Framework?

BR

Answer 1

The fastest way using the serverless framework might be to hook up a lambda function to the s3 event stream, as outlined here.

Then I suppose you could just write to cloudwatch (using console.log) - but likely it'd be simpler to skip cloudwatch and use this lambda function to trigger your step function.

The serverless.yml file should look like this (Additional context taken from below):

  imagewasuploadedevent:
    handler: src/stepfunctions/imageWasUploadedEvent.handler
    events:
     - s3:
        bucket: !Ref AttachmentsBucket
        existing: true

Answer 2

I've used the method described here before: https://docs.aws.amazon.com/eventbridge/latest/userguide/log-s3-data-events.html

In short, you enable Cloudtrail, configure Eventbridge to match/filter the S3 resources and operations you're interested in, and then use Lambda to log those specific events, or can trigger other actions.

Answer 3

The solution was explained in the link above that @Aaton Stuyvenberg pointed

We needed to declare that the bucket existed The event would be an S3 event and the definition of the function should be

  imagewasuploadedevent:
    handler: src/stepfunctions/imageWasUploadedEvent.handler
    events:
     - s3:
        bucket: !Ref AttachmentsBucket
        existing: true

The key existing: true signal that a new bucket does not been to be created More about this is explained here Serverless Framework Documentation