SerializationException when attempting to insert Log into Cloudwatch via API Gateway

Question

We wanted to leverage an existing API Gateway and alter the destination of some errors in an application not hosted in AWS from an RDS to a CloudWatch log group and stream but when testing it I get a SerializationException every time

The model of the data coming into the gateway is

{
  "Message":"foo",
  "StackTrace":"bar",
  "Category": "example"
  "CustomData":{"foo":"bar","fee","fum"},
  "Timestamp": 1564043651175
}

The timestamp was added as it is required to insert a CloudWatch log as shown [here]: https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html

The following is some of the CloudFormation yaml that defines the API Gateway which is making the requests

Uri:
  Fn::Join:
  - ''
  - - 'arn:aws:apigateway:'
    - !Ref AWS::Region
    - ":logs:action/PutLogEvents"
RequestTemplates:
  application/json: !Sub |
    #set($inputRoot = $input.path('$'))
    #set($context.requestOverride.header['X-Amz-Target'] = "Logs_20140328.PutLogEvents")
    #set($context.requestOverride.header['Content-Type'] = "application/x-amz-json-1.1")
    {
      "logGroupName": "${Prefix}-err-group"
      "logStreamName": "${Prefix}-app-errors"
      "logEvents": [
        {
          "message": "message: $inputRoot.Message, stackTrace: $inputRoot.StackTrace, category: $inputRoot.Category, customData: $inputRoot.CustomData",
          "timestamp": "$inputRoot.Timestamp"
        }
      ]
    }

The API method deploys successfully and when testing it in the console using the Method Test and the model above I get the following output from the test

Thu Jul 25 08:14:24 UTC 2019 : Starting execution for request: 364e53c8-aeb4-11e9-91f6-ab8c0812e717
Thu Jul 25 08:14:24 UTC 2019 : HTTP Method: POST, Resource Path: /
Thu Jul 25 08:14:24 UTC 2019 : Method request path: {}
Thu Jul 25 08:14:24 UTC 2019 : Method request query string: {}
Thu Jul 25 08:14:24 UTC 2019 : Method request headers: {}
Thu Jul 25 08:14:24 UTC 2019 : Method request body before transformations: {
    "Message": "Example Message",
    "StackTrace": "Example StackTrace",
    "Category": "Category1",
    "CustomData": {"GUID": "17e332e5-9d03-4e0d-83b1-874f62cb33bb","One": "1","Version": "28.1.1.0","ErrorId": "1212"},
    "Timestamp": 1564040369451 
}
Thu Jul 25 08:14:24 UTC 2019 : Request validation succeeded for content type application/json
Thu Jul 25 08:14:24 UTC 2019 : Request parameter overrides:
Add X-Amz-Target: Logs_20140328.PutLogEvents
Add Content-Type: application/x-amz-json-1.1
Thu Jul 25 08:14:24 UTC 2019 : Endpoint request URI: https://logs.eu-west-2.amazonaws.com/?Action=PutLogEvents
Thu Jul 25 08:14:24 UTC 2019 : Endpoint request headers: {Authorization=<Redacted>, X-Amz-Date=20190725T081424Z, x-amzn-apigateway-api-id=<Redacted>, Accept=application/json, User-Agent=AmazonAPIGateway_<Redacted>, X-Amz-Security-Token=<Redacted>[TRUNCATED]
Thu Jul 25 08:14:24 UTC 2019 : Endpoint request body after transformations: {
  "logGroupName": "test-err-group"
  "logStreamName": "test-app-errors"
  "logEvents": [
    {
      "message": "message: Example Message, stackTrace: Example StackTrace, category, Category1, customData: {GUID=17e332e5-9d03-4e0d-83b1-874f62cb33bb,One=1,Version=28.1.1.0,ErrorId=1212}",
      "timestamp": "1564040369451"
    }
  ]
}

Thu Jul 25 08:14:24 UTC 2019 : Sending request to https://logs.eu-west-2.amazonaws.com/?Action=PutLogEvents
Thu Jul 25 08:14:24 UTC 2019 : Received response. Status: 400, Integration latency: 7 ms
Thu Jul 25 08:14:24 UTC 2019 : Endpoint response headers: {x-amzn-RequestId=3655f4fb-aeb4-11e9-8498-591aad10a10c, Content-Type=application/x-amz-json-1.1, Content-Length=35, Date=Thu, 25 Jul 2019 08:14:23 GMT, Connection=close}
Thu Jul 25 08:14:24 UTC 2019 : Endpoint response body before transformations: {"__type":"SerializationException"}
Thu Jul 25 08:14:24 UTC 2019 : Method response body after transformations: {"__type":"SerializationException"}
Thu Jul 25 08:14:24 UTC 2019 : Method response headers: {X-Amzn-Trace-Id=Root=1-5d3964e0-291c033c3040ef301afc81c5, Access-Control-Allow-Origin=*, Content-Type=application/json}
Thu Jul 25 08:14:24 UTC 2019 : Successfully completed execution
Thu Jul 25 08:14:24 UTC 2019 : Method completed with status: 200

This still occurs when I replace the more complex json request body with a simple string. I found [this post]: Getting SerializationException while trying to PutLogEvents on cloudwatch using golang when someone had the same exception but it was due to a problem with their GoLang rather than going through API

ShipleyCM ShipleyCM · Accepted Answer · 2019-07-29T14:11:14

Turns out I was missing 2 commas after logGroupName and logStreamName. Blinded by looking at it for too long

SerializationException when attempting to insert Log into Cloudwatch via API Gateway

1 Answers