BatchWriteItem operation: The provided key element does not match the schema

Question

I am trying to write csv data from S3 to DynamoDB using AWS Lambda function. I am currently receiving the following error 'BatchWriteItem operation: The provided key element does not match the schema'.

Is there a quick fix to this problem?

import boto3

s3 = boto3.client('s3')
dynamodb = boto3.resource('dynamodb')

def csv_reader(event, context):

    bucket = event['Records'][0]['s3']['bucket']['name']
    key = event['Records'][0]['s3']['object']['key']

    obj = s3.get_object(Bucket=bucket, Key=key)

    rows = obj['Body'].read().split('\n')

    table = dynamodb.Table('customer_id')

    with table.batch_writer() as batch:
        for row in rows:
            batch.put_item(Item={

                'Customer-ID':row.split(',')[0],
                'Name':row.split(',')[1]
            })

Error looks like this:

An error occurred (ValidationException) when calling the BatchWriteItem operation: The provided key element does not match the schema: ClientError Traceback (most recent call last):

File "/var/task/lambda_function.py", line 22, in csv_reader 'Name':row.split(',')[1]

File "/var/runtime/boto3/dynamodb/table.py", line 156, in exit self._flush()

File "/var/runtime/boto3/dynamodb/table.py", line 137, in _flush RequestItems={self._table_name: items_to_send})

File "/var/runtime/botocore/client.py", line 314, in _api_call return self._make_api_call(operation_name, kwargs)

File "/var/runtime/botocore/client.py", line 612, in _make_api_call raise error_class(parsed_response, operation_name)

ClientError: An error occurred (ValidationException) when calling the BatchWriteItem operation: The provided key element does not match the schema

I would expect the data to write from the CSV file to the DynamoDB. Completely new to AWS & Python so any help would be appreciated.

If it says The provided key element does not match the schema, you should provide the schema, and maybe some example input you're using — ChatterOne

Thales Minussi Thales Minussi · Accepted Answer · 2019-05-13T10:30:48

When putting an item you must provide all the mandatory fields, otherwise it will fail with the mentioned exception.

Double check your Table schema and check the mandatory values: at least a Hash Key is mandatory. If you have configured a Hash Key with a Range key, then you must provide both values. Without further information from your table it is difficult for us to guess, but I will try to summarise the problem in the next couple of lines. If it still does not suffice, please update your question with the Table Schema.

So let's say your HashKey is named id, your code should look like this:

batch.put_item(Item={
            'id': 'some-id'
            'Customer-ID':row.split(',')[0],
            'Name':row.split(',')[1]
        })

If a range key is also set and it's named, let's say, test, then your code should look like this:

batch.put_item(Item={
            'id': 'some-id',
            'test': 'some-value',
            'Customer-ID':row.split(',')[0],
            'Name':row.split(',')[1]
        })

If you're not sure how Hash and Range key work together, I suggest you read the docs

BatchWriteItem operation: The provided key element does not match the schema

2 Answers