Publish S3 Bucket Notification to SQS

Question

I am trying to set up my S3 to notify my SQS Queue for a "PUT" Object Creation Event.

I am able to achieve this using CLI by:

aws --profile QA s3api put-bucket-notification --bucket <BUCKET_NAME> --notification-configuration '{ "QueueConfiguration": { "Id": "<EVENT ID>", "Event": "s3:ObjectCreated:Put", "Queue": "<QUEUE ARN>" } }'

Also able to do the same using Java:

NotificationConfiguration notificationConfiguration = new QueueConfiguration(queueArn, EnumSet.of(S3Event.ObjectCreatedByPut));
BucketNotificationConfiguration bucketNotificationConfiguration = new BucketNotificationConfiguration("DropShipInboundQueueDelivery", notificationConfiguration);
client.setBucketNotificationConfiguration(bucketName, bucketNotificationConfiguration)

However when I tried to something similar using CloudFormation template, I cannot find any way to trigger a notification to SQS. The only option I see that works and is documented is to trigger notification to SNS.

I have referred the Cloud Formation Documentation:

I looked at the AWS::S3::Bucket docs to look at the outer syntax. I saw NotificationConfiguration which I need to set
However the Notification Configuration can only contain a list of TopicConfigurations with was the old constructor in JDK before QueueConfiguration was supported

I tried doing something like this:

"NotificationConfiguration" :{
    "QueueConfiguration": {
        "Id": "DropshipInboundEventNotification",
        "Event": "s3:ObjectCreated:Put",
        "Queue": "arn:aws:sqs:*:*:Dropship-Inbound-qa"
    }
},

But this as expected threw an error: "Encountered unsupported property QueueConfiguration" from amazon.

Looked at this API documentation

I would like to know if someone has been able to do this using CloudFormation Templates as thats how I am maintaining all the other AWS resources and do not want to do anything special for this particular feature.

Any help is appreciated.

I haven't tried this feature myself, but I've found CloudFormation in general to lag behind the API on the order of months. If there was a change some time back as to how this works, it's likely it just hasn't migrated to CF... yet? — DanielM

Kerim Doruk Akıncı Kerim Doruk Akıncı · Accepted Answer · 2015-10-21T15:55:01

There is no need "Id" in Cloudformation Template ( You can check from QueueConfiguration Doc ) and your second mistake, that is not "QueueConfiguration", it's "QueueConfigurations". Because of that you get an error that says "Encountered unsupported property QueueConfiguration"

It must be something like that.

"S3Bucket":{
   "Type" : "AWS::S3::Bucket",
   "Properties" : {
      "AccessControl" : String,
      "BucketName" : String,
      "CorsConfiguration" : CORS Configuration,
      "LifecycleConfiguration" : Lifecycle Configuration,
      "LoggingConfiguration" : Logging Configuration,
      "NotificationConfiguration" :

{ "QueueConfigurations" : [ {
  "Event" : "s3:ObjectCreated:Put",
  "Queue" : "arn:YOURQUEUEARN"
} ] },

      "Tags" : [ Resource Tag, ... ],
      "VersioningConfiguration" : Versioning Configuration,
      "WebsiteConfiguration" : Website Configuration Type
   }
}

While you are reading cloudformation template documents, you must be careful about "Required:" sections. If it is not required, you don't need to fill it, just remove that line from your template if you don't use it( Like S3 Tags ).

Other Docs about it:

S3BucketDocs

NotificationConfigurationDocs

Publish S3 Bucket Notification to SQS

1 Answers