How can I configure my EC2 instance running DPDK to filter traffic between elastic IP and another ec2 instance?

Question

I have a hardware setup that I need to simulate on AWS. In hardware, I have a customer's computer connected to the internet via a cable modem. In between the cable modem and the customer's computer I insert my computer running DPDK and a packet filter application. All packets from the cable modem enter my computer/dpdk at Int-1, are processed, and leave my computer on Int-2 to go to the customer's system. The same data path is traversed in reverse for packets originating from the customer's system. Packets simply follow the ethernet cables to where they are supposed to go.

I need to replicate that in the AWS cloud, but do not have ethernet cables to force the routing of packets. I need to insert my EC2 instance running DPDK between an Elastic IP and the customer's EC2 instance with a private IP. The setup looks like this:

                  VPC
                 +------------------------------------------------+
                 |                                                |
                 |    c5.2xlarge EC2           t2.micro EC2       |
                 |   +--------------------+   +----------------+  |
                 |   | My ec2 with DPDK   |   | Customer ec2   |  | 
     EIP 1.2.3.4 <---> Int-1 10.0.1.101   |   |                |  | 
                 |   |        ^           |   |                |  | 
                 |   |        |           |   |                |  | 
                 |   |        v           |   |                |  | 
                 |   |  <processing>      |   |                |  | 
                 |   |        ^           |   |                |  | 
                 |   |        |           |   |                |  | 
                 |   |        v           |   |                |  | 
                 |   |   10.0.2.101 Int-2 <---> 10.0.1.89 eth0 |  | 
                 |   |                    |   |                |  | 
                 |   +--------------------+   +----------------+  | 
                 +------------------------------------------------+

This is running on centos7. When DPDK is running ens6 becomes Int-1, and ens7 becomes Int-2.

The EIP 1.2.3.4 used to be attached to the customer's private IP 10.0.1.89, so internet users had access to the customer's ec2, and the customer's ec2 users had access to the internet.

After my ec2 instance is added to the VPC, and the EIP is detached from the customer's ec2 and reattached to my ec2, now I want to filter traffic in both directions through to and from the customer's ec2.

If my ec2 was not running DPDK I could simply use iptables to NAT traffic in both directions. But with DPDK I need a user-space NAT that runs on my ec2, or I need some other way to route packets from the EIP to my Int-1 interface, and then out the Int-2 interface to the customer ec2, and back.

There are many purported DPDK tcp/ip stacks out there, but none really seem to work for one reason or another. I would love to make this work with AWS routing alone, and no NAT, but don't know if that is possible.

Help!

Andriy Berestovskyy Andriy Berestovskyy · Accepted Answer · 2019-07-10T21:38:35

To implement a basic NAT you don't need a TCP/IP stack. Just parse each frame down to the IP header, and substitute any IP 1.2.3.4 to 10.0.1.101 and vice-versa. Then just set the mbuf.ol_flags to recalculate the checksums in the NIC or do it in software and you're done.

Please see the Mbuf library and rte_ipv4_udptcp_cksum() for more details regarding the checksums.

Another issue is that your DPDK filtering application works as an L3 device (i.e. router), while it might be much simpler if it worked as a transparent L2 device (i.e. transparent bridge). This will eliminate the need of extra route on the gateway.

How can I configure my EC2 instance running DPDK to filter traffic between elastic IP and another ec2 instance?

1 Answers