AWS - Network Performance doesn't correspond to what it should be as mentioned by aws amazon

Question

Good morning everybody,

I am really trying to understand about AWS Network Performance.

I did a lot of tests and my conclusion is that the speed really doesn't correspond to what it should be as mentioned by AWS amazon.

Well, I created 9 different EC2 on this test. All these instances were created on the same zone (North Virginia), all running on Ubuntu 18.04. ENA: Yes. AMI - Ubuntu 18.04

Measurement: Speedtest cli

1st instance: t2.micro (low to moderate) > Down: 488.86Mbps/ Up: 946.49Mbps
2st instance: t2.large (low to moderate) > Down: 940.31Mbps/ Up: 900.47Mbps
3rd instance: t2.xlarge (moderate) > Down: 964.90Mbps/ Up: 942.10Mbps
4th instance: t3.micro (Up to 5 Gigabit) > Down: 4002.90Mbps/ Up: 2844.04Mbps
5th instance: t3a.nano (Up to 5 Gigabit) > Down: 1584.52Mbps/ Up: 1451.76Mbps
6th instance: t3a.small (Up to 5 Gigabit) > Down: 2758.09Mbps/ Up: 2143.01Mbps
7th instance: t3.2xlarge (Up to 5 Gigabit) > Down: 2936.57/ Up: 2733.28 Mbps
8th instance: m5a.large (Up to 10 Gigabit) > Down: 3178.89/ Up: 2291.97Mbps
9th instance: r5n.large (Up to 25 Gigabit) > Down: 4361.71/ Up: 3383.65Mbps

These measurement results:

Instances 1-2

Instances 3-4

Instances 5-6

Instances 7-8

Instance 9

Having these results, we can conclude:

• Network Performance:
  

1. Low to Moderate: The best we can get is almost 1Gigabit. (Down and Up) Very good speed for low to moderate.
   
   
2. Moderate: Keep about almost 1Gigabit network performance. That's fine
   
   
3. Up to 5 Gigabit: We can get down/up starting on about 1.5 Gigabit. The fastest I measured was down/up 4002Mbps/2844Mbps. Great, the network performance on "up to 5 gigabit" is corresponding perfectly.
   
   
4. Up to 10 Gigabit: Download and upload the same as an instance running "up to 5 gigabit". It's not corresponding to what it should be.. 9 gigabit at least should be fine!
   
   
5. Up to 25 Gigabit: I can't believe. Download and upload network performance practically the SAME as "up to 5 gigabit". It should be at least 20-22 Gigabit.
   
   
6. I'm almost sure that "25 Gigabit, 50 Gigabit, 75 Gigabit, 100 Gigabit will happen the same. It won't connect more than 4 Gigabit.
   

Could anyone explains what is going on? Is that normal? Do I need to make any settings to enhance network performance?

Hope to hear from the community, Thanks a lot!!

Mat

Answer 1

There appear to be several potential ssues at play, here.

Perhaps the most noteworthy is that the performance numbers in the specs are not Internet bandwidth, they're Ethernet bandwidth.

Network traffic to the Internet is limited to 5 Gbps (full duplex).

https://aws.amazon.com/ec2/instance-types/

You may counter that your destination is not "The Internet" because your test target is a system located in the same AWS region, but the traffic is still going through your VPC's Internet Gateway.

The next consideration is that the maximum performance potential for EC2 applies only within the same VPC, within a placement group, using private IP addresses, with multiple simultaneous traffic flows.

Traffic to and from EC2 instances in the same or different Availability Zones within a region can now take advantage of up to 5 Gbps of bandwidth for single-flow traffic, or 25 Gbps of bandwidth for multi-flow traffic (a flow represents a single, point-to-point network connection) by using private IPv4 or IPv6 addresses as described here.

https://aws.amazon.com/blogs/aws/the-floodgates-are-open-increased-network-bandwidth-for-ec2-instances/

Before you can use a tool like this for benchmarking, you also need to be able to verify that the tool would, e.g., actually be able to accurately test the range of capabilities in question.

Speed test tools require a server on the other end that is capable of generating enough data to saturate the connection, and you presumably don't have a way of verifying that, though you might inquire about the tester's actual capabilities with Ookla.

You also need to verify that the tool isn't exhausting some other resource on the server, such as CPU, which might not manifest itself as 100% CPU utilization because the number of saturated cores won't exceed the number of threads running.

See https://aws.amazon.com/premiumsupport/knowledge-center/network-throughput-benchmark-linux-ec2/ for bemchmarking guidelines, including using iperf on two machines to test the bandwidth between them.