Try to upload bulk 200K record in DynamoDB table using CLI command

0
votes

CLI Command: $ aws dynamodb batch-write-item --request-item file://db_mobile_block_1.json

Getting Error: An error occurred (ValidationException) when calling the BatchWriteItem operation: 1 validation error detected: Value '{uns-prod-blocked-mobile-numbers= [ WriteRequest (putRequest=PutRequest(item={mobile=com.amazonaws.dynamodb.v20120810.AttributeValue@bbfde161}, workloadProfileName=null), deleteRequest=null), WriteRequest (putRequest=PutRequest(item={mobile=com.amazonaws.dynamodb.v20120810.AttributeValue@2e6bc812}, workloadProfileName=null), deleteRequest=null),

WriteRequest (putRequest=PutRequest(item={mobile=com.amazonaws.dynamodb.v20120810.AttributeValue@850aefcc}, workloadProfileName=null), deleteRequest=null), WriteRequest (putRequest=PutRequest(item={mobile=com.amazonaws.dynamodb.v20120810.AttributeValue@63c269ec}, workloadProfileName=null), deleteRequest=null) ] }' at 'requestItems' failed to satisfy constraint: Map value must satisfy constraint: [Member must have length less than or equal to 25, Member must have length greater than or equal to 1]

1
amazon-dynamodbcommand-line-interfacebulkinsert
I try to upload only 1109 records at a time but getting same error. - Vivek Karande
BatchWrite has a limit of 25 items per request. You should split your JSON into smaller chunks before sending a request - Vlad Holubiev
Hi @VladHolubiev I need to insert 200000 records it's very difficult and time taking split into 25 items chunks. Can we able to increase the limit up to 10000 items? - Vivek Karande
@vivek. In a batchwrite you can pass maximum 25 items. One way to speedup the execution is to group items into multiple batch of 25 items and do parallel excecution. Make sure you have configured enough WCU. - Libu Mathew

1 Answers

0
votes

What you are doing in this request is not to to upload a "200K record", but use the BatchWriteItem request to upload many (200,000?) different records. You can't do that - BatchWriteItem has a hard limit of 25 items. You'll just have to split your items to batches of up to 25 items and send separate requests - in sequence or in parallel.

Alternatively, if you really want to insert a single record which is 200KB in size, then you are doing it wrong. You should use PutItem, not BatchWriteItem, and insert a single record.