AWS-SDK DynamoDB NodeJS bad performance?

Question

I'm testing the dynamoDB API with the AWS-SDK for NodeJS. But I have some strange behavior, dynamoDB seems to be slower to respond and I don't understand why.

From the AWS Console, I set "100" Read capacity units to "MyTable"

I made some benchmarks :

For 10 calls the services seems to respond in less then 1 sec; 100 calls (1,5 sec), 1000 calls (4,5 secs), 5000 calls (30 secs)

See below for my code :

var http = require('http');
var https = require('https');
var AWS = require("aws-sdk");

var start = new Date();

http.globalAgent.maxSockets = 100000;
https.globalAgent.maxSockets = 100000;

AWS.config.update({
  accessKeyId: 'MyAccessKey',
  secretAccessKey: 'MySecretKey',
  region: 'MyRegion'
});

var dynamodbDoc = new AWS.DynamoDB.DocumentClient();

var startedDate = new Date()
console.log('Start');
for (var i = 0; i < 5000; i++) {
    getVolume("604275e8-17de-4582-bbc2-1b5eaf3b0a4c");
}   
console.log('Over ' + (new Date() - startedDate) + ' ms');

function getVolume(key) {
    var params = {
        TableName: "MyTable",
        KeyConditionExpression: "#id = :key",
        ExpressionAttributeNames: {
            "#id": "id"
        },
        ExpressionAttributeValues: {
            ":key": key
        }
    };
    var requestStarted = new Date()
    dynamodbDoc.query(params, function (err, data) {
        if (err) {
            console.error("Unable to query. Error:", JSON.stringify(err, null, 2));
        } else {
            console.log("DynamoDB : " + (new Date() - requestStarted));
        }
    });
}

With the 5000 calls, the "for" statement is over after 3 seconds but the first dynamoDB respond after 30 seconds, why ?

Thank you in advance,

Mark B Mark B · Accepted Answer · 2016-01-06T15:06:25

You are sending 5000 calls to DynamoDB virtually simultaneously. First, there is probably some CPU and network bottlenecks on the computer running the test keeping this from performing as well as you want. Second, if this is really what you intend to be testing then you would need to increase the write capacity of DynamoDB to something much higher.

If you are trying to perform the calls to DynamoDB one at a time, then you need to wait for the callback to finish before executing the next call.

AWS-SDK DynamoDB NodeJS bad performance?

2 Answers