Add an attribute to every item in a DynamoDB table with millions of items

Question

I am pretty new to AWS, but I need to add an attribute (and set the value to something) to every single item in a DynamoDB table.

I'm trying to write a generic script that will be able to do this to any table passed into it.

After a little bit of digging, it looks like I should be able to do this just by doing a scan of the table, getting the items, and then updating every single item.

My questions are: (1) Does this seem like a reasonable approach? (2) Is there an easier / better way to do this automatically? (a table-wide addition of an attribute to every item in the table?)

there is no easier approach, however the given approach will also have issues, like what if application is also trying to write to the table? Another thing that you can do is instead of readily updating all the items update the items as and when they are requested. — best wishes
Would you happen to know what the default behavior is if another application is trying to write to the table at the same time the scan and update is happening? Will the one that gets to the item first succeed, but then the second one trying to do the update will fail? — CustardBun
docs.aws.amazon.com/amazondynamodb/latest/developerguide/… — best wishes

rocketlobster rocketlobster · Accepted Answer · 2020-03-25T23:48:44

Yes it sounds reasonable, not ideal, but reasonable
Sure, there is always a better way, but this is an easy way, Here is a snippet I ran as a local JS file...

const AWS = require('aws-sdk');
const dynamoDb = new AWS.DynamoDB.DocumentClient({
    region:'us-east-1'
});

async function main(event, context) {
    let tableContents;
    try{
        //get items from dynamo
        const params = {
            TableName: `${YOUR_TABLE_NAME}`,
        };
        tableContents = await scanDB(params);
    }catch(err){
        console.log(err);
        return err;
    }
    let calls = [];
    tableContents.forEach(function(value){
        let params = {
            ExpressionAttributeValues: {
                ":newAttribute": false,
            },
            Key: {
                "indexKey": value.indexKey
            },
            TableName: `${YOUR_TABLE_NAME}`,
            UpdateExpression: "SET newAttribute = :newAttribute",
            };
        calls.push(dynamoDb.update(params).promise());
    });
    let response;
    try{
        response = await Promise.all(calls);
    }catch(err){
        console.log(err);
    }
    return response;
}
async function scanDB(params) {
    let dynamoContents = [];
    let items;
    do{
        items =  await dynamoDb.scan(params).promise();
        items.Items.forEach((item) => dynamoContents.push(item));
        params.ExclusiveStartKey  = items.LastEvaluatedKey;
    }while(typeof items.LastEvaluatedKey != "undefined");
    return dynamoContents;
};
main();

Good luck!

Add an attribute to every item in a DynamoDB table with millions of items

1 Answers