I am looking to implement global rate limiting to a production deployment on Azure in order to ensure that my application do not become unstable due to an uncontrollable volume of traffic(I am not talking about DDoS, but a large volume of legitimate traffic). Azure Web Application Firewall supports only IP based rate limiting.
I've looked for alternatives without to do this without increasing the hop count in the system. The only solution I've found is using limit_req_zone
directive in NGINX. This does not give actual global rate limits, but it can be used to impose a global rate limit per pod. Following configmap is mounted to the Kubernetes NGINX ingress controller to achieve this.
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-ingress-ingress-nginx-controller
namespace: ingress-basic
data:
http-snippet : |
limit_req_zone test zone=static_string_rps:5m rate=10r/m ;
location-snippet: |
limit_req zone=static_string_rps burst=20 nodelay;
limit_req_status 429;
static_string_rps
is a constant string and due to this all the requests are counted under a single keyword which provides global rate limits per pod.
This seems like a hacky way to achieve global rate limiting. Is there a better alternative for this and does Kubernetes NGINX ingress controller officially support this approach?(Their documentation says they support mounting configmaps for advanced configurations but there is no mention about using this approach without using an additional memcached
pod for syncing counters between pods)
https://www.nginx.com/blog/rate-limiting-nginx/#:~:text=One%20of%20the%20most%20useful,on%20a%20log%E2%80%91in%20form. https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#global-rate-limiting