AWS Lambda JSON to DynamoDB KeyError Serializing JSON

Question

I am setting up my lambda function on AWS and want to insert my JSON into the DynamoDB (NoSQL). I think it has to do with the serialization of the JSON (decimal_serializer), but I might be wrong. Thanks for help!

I get following error while serializing this data:

Error MSG:

{
  "errorMessage": "'male_confidence'",
  "errorType": "KeyError"
}

JSON DATA to INSERT:

{
  "device_id": "abc876",
  "recorded_at": "1496136878",
  "customers": [
    {
      "male_confidence": "0.2",
      "female_confidence": "0.8"
    },
    {
      "male_confidence:": "0.1",
      "female_confidence": "0.9"
    }
  ]
}

Lambda Function Handler

import boto3
import json

def lambda_handler(event, context):
    # TODO implement
    client = boto3.client('dynamodb')
    for customer in event['customers']:
        client.put_item(TableName="cv_data_1", Item={'device_id': {"S": event['device_id']}, 'male_confindence': {"N": customer['male_confidence']}, 'female_confidence': {"N": customer['female_confidence']}, "timestamp":{ "N": event['recorded_at']}})
    print('Successfully processed %s items.' % str(len(event['customers'])))

AWS Output Log Error (Detail):

During handling of the above exception, another exception occurred:

09:23:38
Traceback (most recent call last):

09:23:38
File "/var/runtime/awslambda/bootstrap.py", line 463, in <module>

09:23:38
main()

09:23:38
File "/var/runtime/awslambda/bootstrap.py", line 459, in main

09:23:38
handle_event_request(request_handler, invokeid, event_body, context_objs, invoked_function_arn)

09:23:38
File "/var/runtime/awslambda/bootstrap.py", line 240, in handle_event_request

09:23:38
result = to_json(result)

09:23:38
File "/var/runtime/awslambda/bootstrap.py", line 215, in to_json

09:23:38
return json.dumps(obj, default=decimal_serializer)

09:23:38
File "/var/lang/lib/python3.6/json/__init__.py", line 238, in dumps

09:23:38
**kw).encode(obj)

09:23:38
File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode

09:23:38
chunks = self.iterencode(o, _one_shot=True)

09:23:38
File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencode

09:23:38
return _iterencode(o, 0)

09:23:38
File "/var/runtime/awslambda/bootstrap.py", line 104, in decimal_serializer

09:23:38
raise TypeError(repr(o) + " is not JSON serializable")

09:23:38
TypeError: <FrameSummary file /var/task/lambda_function.py, line 8 in lambda_handler> is not JSON serializable

From the error message, it appears that themale_confidence is missing in one of the customer. (No pun intended). Can you just print the event in the lambda handler so that you can see what data event contains? The output will be in cloudwatch logs. — user1741851
{'device_id': 'abc876', 'recorded_at': '1496136878', 'customers': [{'male_confidence': '0.2', 'female_confidence': '0.8'}, {'male_confidence:': '0.1', 'female_confidence': '0.9'}]} , but if I output print(customer)in the loop it returns me the dictsee below. — zer02
I printed customer['male_confidence'] in the loop and it returns me 0.2, the second loop breaks with the error above. — zer02
{'device_id': 'abc876', 'recorded_at': '1496136878', 'customers': [{'male_confidence': '0.2', 'female_confidence': '0.8'}, {'male_confidence:': '0.1', 'female_confidence': '0.9'}]} HELLO {'male_confidence': '0.2', 'female_confidence': '0.8'} ##### 0.2 0.8 HELLO {'male_confidence:': '0.1', 'female_confidence': '0.9'} 'male_confidence': KeyError Traceback (most recent call last): File "/var/task/lambda_function.py", line 14, in lambda_handler mc = customer['male_confidence'] KeyError: 'male_confidence' — zer02

Adam Owczarczyk Adam Owczarczyk · Accepted Answer · 2017-05-30T13:04:20

Typo:

  "male_confidence:": "0.1",
  "female_confidence": "0.9"

"male_confidence:"

AWS Lambda JSON to DynamoDB KeyError Serializing JSON

2 Answers