AWS SNS metadata when pushing to SQS

Question

I'm using .net sdk to push messages to SNS which then publishes messages to subscribed SQS Queues, but SNS adds metadata such as message type etc, and i end up with larger JSON payload which i do not want, especially when SQS is limited to 256k per message.

Is it possible to send a command to SNS and tell it not to send that metadata?

Here's the code:

var credentials = new StoredProfileAWSCredentials("dev");

using (var client = new AmazonSimpleNotificationServiceClient(credentials, Amazon.RegionEndpoint.EUWest1))
{
    var request = new PublishRequest();
    request.Message = message;
    request.TargetArn = "ARN";

    var response = client.Publish(request);
    // return response;
}

This is what SNS pushes to SQS:

{  
   "Type":"Notification",
   "MessageId":"3333333",
   "TopicArn":"ARN",
   "Message": {
       "Id":45,
       "MessageId":"a871f8d7-7091-4548-87c5-da3bb4131044",
       "Payload": {
           "Internalid":"a7c50558-fc9c-47a6-b0ed-cb0cf5020f55",
           "MessageId":"a871f8d7-7091-4548-87c5-da3bb4131044",
           "TravelersName":"aaa aaaa",
           "TravelersEmail":"[email protected]",
           "OwnersEmail":null,
           "AccommodationId":1,
           "BookingCode":"a523cd07-5506-48ad-9380-2e94002722e0"
       },
       "EventName":"BookingRecordCreated",
       "CreatedOnUtc":"2016-09-26T15:14:55.437",
       "HandledOnUtc":null
    },
    "Timestamp":"2016-09-26T15:14:50.923Z",
    "SignatureVersion":"1",
    "Signature":"dfgdfg///==",
    "SigningCertURL":"https://sns",
    "UnsubscribeURL":"https://sns."
}

and this is what i publish to SNS

"Message":{
   "Id":45,
   "MessageId":"a871f8d7-7091-4548-87c5-da3bb4131044",
   "Payload": {
       "Internalid":"a7c50558-fc9c-47a6-b0ed-cb0cf5020f55",
       "MessageId":"a871f8d7-7091-4548-87c5-da3bb4131044",
       "TravelersName":"aa aaa",
       "TravelersEmail":"[email protected]",
       "OwnersEmail":null,
       "AccommodationId":1,
       "BookingCode":"a523cd07-5506-48ad-9380-2e94002722e0",
       "EventName":"BookingRecordCreated",
       "CreatedOnUtc":"2016-09-26T15:14:55.437",
       "HandledOnUtc":null
    }
}

E.J. Brennan E.J. Brennan · Accepted Answer · 2016-09-27T08:51:14

You want to enable RAW Message delivery in SNS:

http://docs.aws.amazon.com/sns/latest/dg/large-payload-raw-message.html

In addition to sending large payloads, with Amazon SNS you can now enable raw message delivery for messages delivered to either Amazon SQS endpoints or HTTP/S endpoints. This eliminates the need for the endpoints to process JSON formatting, which is created for the Amazon SNS metadata when raw message delivery is not selected. For example when enabling raw message delivery for an Amazon SQS endpoint, the Amazon SNS metadata is not included and the published message is delivered to the subscribed Amazon SQS endpoint as is.

AWS SNS metadata when pushing to SQS

2 Answers