ValidationException while doing PutItem: Missing the key in the item: ClientError

Question

I have configured an AWS Lambda in Python 2.7 to read event from Firehose Delivery Stream and writing to a DynamoDB table 'My_Tab' with attribute 'element_class' (type String) as partition key

import json
import boto3

def lambda_data_handler(event, context):

    dynamodb = boto3.resource('dynamodb', region_name='ap-south-1')
    table = dynamodb.Table('My_Tab')

    response = table.put_item(Item = event)
    print(json.dumps(response))
    print("Row-" + str(index) + " written to DynamoDB successfully")

For streaming to the Firehose, I am binary encoding a JSON file my_data.json and then sending the data using AWS CLI put-record utility as follows:

c:\Program Files\Amazon\AWSCLI>aws firehose put-record --delivery-stream-name My_Dlv_Stream --record file://C:/Users/somnath/my_data.json
{
    "RecordId": "DvH2dm5W75F9+bwjJesUW8FoPqQZJOF66etwGoWUycMX..."
}

The JSON file my_data.json is having a single JSON record as follows:

{"Data":"{\"element_class\":\"1001\"}\n"}

But the data is not getting written to the DynamoDB table My_Tab with the below CloudWatch error log:

An error occurred (ValidationException) when calling the PutItem operation: One or more parameter values were invalid: Missing the key element_class in the item: ClientError
Traceback (most recent call last):
File "/var/task/lambda_KFH_2_DynDB.py", line 21, in lambda_data_handler
response = table.put_item(Item = event)
File "/var/task/boto3/resources/factory.py", line 520, in do_action
response = action(self, *args, **kwargs)
File "/var/task/boto3/resources/action.py", line 83, in __call__
response = getattr(parent.meta.client, operation_name)(**params)
File "/var/task/botocore/client.py", line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/var/task/botocore/client.py", line 612, in _make_api_call
raise error_class(parsed_response, operation_name)
ClientError: An error occurred (ValidationException) when calling the PutItem operation: One or more parameter values were invalid: Missing the key element_class in the item

Add a print(json.dumps(event)) at the beginning of function. Does your event has a top level key element_class? — FelixEnescu
Yes my event has a top-level key called 'element_class'. It will have a String value — somnathchakrabarti
Can you please add the output of print(json.dumps(event)) from logs? — FelixEnescu

somnathchakrabarti somnathchakrabarti · Accepted Answer · 2018-07-10T07:01:40

Just added the below code to extract data from the recordId of records attribute from the event, instead of taking the event directly. And it worked.

    it = json.loads(event['records'][0]['data'])
    response = table.put_item(Item = it)

@blueCat : Thanks for pointing to printing the event. The data format I was expecting was wrong.

ValidationException while doing PutItem: Missing the key in the item: ClientError

1 Answers