Handling nested JSON messages with AWS IoT Core rules and AWS Lambda

Question

We are using an AWS IoT Rule to forward all messages from things to a Lambda function and appending some properties on the way, using this query:

SELECT *, topic(2) AS DeviceId, timestamp() AS RoutedAt FROM 'devices/+/message'

The message sent to the topic is a nested JSON:

{
   version: 1,
   type: "string",
   payload: {
     property1: "foo",
     nestedPayload: {
         nestedProperty: "bar"
    }
  }
}

When we use the same query for another rule and route the messages into an S3 bucket instead of a Lambda, the resulting JSON files in the bucket are as expected:

{
   DeviceId: "test",
   RoutedAt:1618311374770,
   version: 1,
   type: "string",
   payload: {
     property1: "foo",
     nestedPayload: {
         nestedProperty: "bar"
    }
  }
}

But when routing into a lambda function, the properties of the "nestedPayload" are pulled up one level:

{
   DeviceId: "test",
   RoutedAt:1618311374770,
   version: 1,
   type: "string",
   payload: {
     property1: "foo",
     nestedProperty: "bar"
  }
}

However, when debugging the Lambda locally using VS Code, providing a JSON file (in other words: not connecting to AWS IoT Core), the JSON structure is as expected, which is why I am assuming the error is not with the JSON serializer / deserializer, but with the rule.

Did anyone experience the same issue?

Schweder Schweder · Accepted Answer · 2021-04-13T11:57:27

It turns out, the issue was with the SQL version of the rule.

We created the rule routing to the Lambda using CDK, which by default set the version to "2015-10-08". The rule routing to S3, which didn't show the error, was created manually and used version "2016-03-23". Updating the rule routing to the Lambda to also use "2016-03-23" fixed the issue.

Handling nested JSON messages with AWS IoT Core rules and AWS Lambda

1 Answers