I'm looking for a fast stateful firewall in Linux that may process lots of packets and send some of them to external programs.
Is it possible to make netfilter (iptables, nftables) working with Intel DPDK in order to improve performance?
DPDK has a "Kernel Network Interface" that works somewhat like a tap/tun device, but is zero-copy. Using these, you can do your RX/TX in userspace and then pass packets into the linux network stack. I don't know if this would buy you much performance aside from possibly removing some IRQ overhead.
Some other options, if you're not married to netfilter, could be to use one the userspace BSD network stacks (e.g. libuinet) or a rump kernel on top of DPDK.
You need to deploy an acceleration software stack from 6WIND in order to keep using iptables, iproute2, etc. DPDK does not provide networking stacks.
Assuming you are using Linux kernel iptables, route entries, IPsec entries, etc. using iptables, setkey, iproute2, etc. without modifications ; then 6WIND's software stacks offload the packet processing into their acceleration stacks.
So you can keep you Linux admin tools (web interface, Openstack, etc.) while 6WIND software stack takes care of delivering the performance. It works great on DPDK and other CPUs too like Tilera, Cavium Octeon, Broadcom XLP, ...
See their website for more information.
I had tried DPDK KNI to utilize iptables in Linux netfilter. It worked, but the overall performance was a little less than pure-kernel iptalbes.
According to my simple NAT experiment, iptables provided 290Mbps over 1G link for 64 bytes packets. And iptables with KNI showed 240Mbps.
Although DPDK KNI runs in PMD mode and eliminates interrupts, it does not increases the bandwidth since Linux kernel also adopts NAPI(New API) which changes run-time mode from interrupt to polling and vice verse.
When I ran my simple DPDK application for NAT, it showed 761Mbps that is almost the maximum bandwidth for 64 bytes packets in theory.
I think softirq in Linux kernel consumes much of the time to process packets.