How to monitor HTTP status codes from nginx ingress on AKS?

0
votes

I'm trying to setup alerting based on http status codes in python web server (fastapi) which runs on Azure Kubernetes Services (AKS) behind an nginx ingress controller. AKS has integration with azure log workspace enabled, and i can see the logs there, but I can't find any example query, or even any example of status code metric available. After looking at the docs, it seemed to me that i have to instrument it with application insights (c.f. tracing) to get status codes. It seemed to me that this is somewhat excessive (although useful, of course).

In other cloud providers I was always able to find the http status code metrics on load balancers and use them to alert if the ratio of errors started growing. But I can't find any straightforward to do the same with AKS. Any ideas are welcome. I would prefer to use the tooling provided by Azure itself, e.g. not some other vendor.

1
azure-aksazure-monitoring

1 Answers

1
votes

I don't think that you will find the HTTP status somewhere in the tables created by Azure Monitor for Container. I think that your best bet would be to enable the HTTP & HTTPS traffic logs on the Nginx Ingress if it is not already done. I think that the HTTP code will be there by default, but if you want something easier to query in Log Analytics, you could for example change the Nginx log format to something like

status=$status request=$request ...

and then use a simple query for your alert :

ContainerLog 
 | where LogEntry contains "status=500"