DynamoDB REST API Possible Write Throughput

Question

I'm currently running some performance tests using NodeJS and DynamoDB. Especially I'm interested in the possible write throughput of one NodeJS process which persists items to DynamoDB.

I ran my tests on an EC2 m1.medium instance. To access DynamoDB I use the dynamo NodeJS module. For the duration of the tests I provisioned a DynamoDB table with 1000 write capacity units. I was persisting items of about 700 bytes size.

The result of these tests was that the max. write throughput with this system is at about 300 items/second. At this throughput the virtual CPU of the EC2 instance is at nearly 100%.

Having in mind that calling the API is basically calling a HTTP REST service the result looks kind of realistic compared to this post: What is the performance for Node.js' http.request ? How many concurrent request it can handle?

Can anyone confirm my test result or did you experience different results? Has anyone tested the AWS DynamoDB JAVA or C# SDKs? What performance do you get there?

UPDATE: The 300 items/seconds were with Nodetime enabled. Without that the limit is at about 780 items/second which pushed to the DynamoDB REST API.

Nodetime has some overhead when profiling is active (i.e. web console is open), but it shouldn't be so high. Could you post or send me the script you're doing your benchmarks with if applicable. I'd really like to understand what causes that overhead. Thank you! (I'm creator of Nodetime). You could send it to support email, which can be found at nodetime.com in the footer. — logix
BTW, I'd recommend pausing Nodetime's profiler when doing benchmarks. You'll still see the metrics in web console after the benchmark, because metrics are aggregated and sent always. So basically you would call nodetime.pause() after calling profile() and not open web console at nodetime.com until you're done with the benchmark. — logix

bwight bwight · Accepted Answer · 2012-08-21T20:43:13

It seems like your bottleneck is the m1.medium instance. Having not used node.js much I can't say whether another option would perform better but from what I node.js is one of the more lightweight choices you could use. Keep in mind that the REST calls to AWS Services like DynamoDB are not just simple REST calls, they need to create a signature for each request which requires some basic hashing and cryptography which could be why the CPU is behaving the way it is. It might not seem like a lot to hash a small string but when you have to do it 300 times per second it can easily kill the CPU.

My recommendations are to use another instance type, for example the c1.medium will give you 150% more power over the m1.medium. In addition to this you can also try to write a simple test script in python or php to see how the results compare with node.js.

DynamoDB REST API Possible Write Throughput

1 Answers