IOPS limits on EC2 with EBS mounting

1
votes

I had posted a question on IOPS limit where in EC2 instance does not use EBS. This question deals with EC2 instance with Elastic Block storage. The environment goes as -

  • Two c4.8xlarge EC2 instances, one c4.8xlarge instance supports 500 MB/s throughput
  • Connection pool is 2049
  • Got rid of DNS Cache
  • EBS storage of 120 GB with each instance
  • Application is load tested with JMeter 2.13 for one hour on distributed mode using 10 m4.4xlarge instance
  • Test is executed with 700 threads with ramp up period of 30 mins and then continues with all 700 threads for 30 mins.
  • There are two APIs being load tested one has Write Operation (Post Request) and other had Read Operation (Get Request)
  • Test reaches throughput of 7800 requests/sec for read operation and 300 requests/sec for write operation with 400 threads and stays at ~7800 requests/sec irrespective of the threads increasing up to 700 threads.
  • Following are the graphs of Read and Write Throughput during tests for EBS -

enter image description here enter image description here

  • And this is how average Read and Write size appear to be during test - enter image description here

Considering graphs above could I safely assume that I am not hit by IOPS limit?

1
amazon-web-servicesamazon-ec2jmeteramazon-ebs
7800 requests per second. Each request is 60 kB so total is 468000 KB which is around 460 MB/sec. for c4.8xlarge it is 500 MB per second is the max as per this: docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-ec2-config.html – phoenix

1 Answers

1
votes

7800 requests per second. Each request is 4 KB for read. So, total is 31200 KB, which is around 31 MB/sec. For write it comes around 18 MB/sec as per your data. For c4.8xlarge 500 MB per second is the max as per this: EBS EC2 Config.

The throughput depends on the EBS volume size. The current ratio is 30:1 as per: EBS IOPS:GB ratio.

So, for 120 GB on c4.8xlarge, you need to get 3600 Mbps or 450 MB/sec.

Also, did you try increasing the number of m4.4xlarge instances or using a much powerful m4.10x instances. May be that can also be a bottleneck. What was the CPU usage on these instances when the tests were running?