Setup
- Google Container Engine (kubernetes)
- Deployment / pod with my web server application (Torando/python)
- Ingress for web server service in kubernetes - which created load balancer in GCP
- Backend service in load balancer with a backend which is the web server
- Frontend which directs to the backend server
- Host and path rules directing custom domain and subdomains to relevant backend
- Firewall rules set as created by ingress
So...
When I created all of the above, I created a new TCP health check with the correct port and everything. After a few minutes that it updates, everything seems to work ok - 3/3 healthy instances in the load balancer and my backend is exposed via the frontend perfectly. I need only TCP health check, not HTTP.
Problem
For some reason, after a few minutes, the health check configuration keeps resetting to the default HTTP health check, that doesn't work, leaving the gateway broken, and any request returns: Error: Server Error
The server encountered a temporary error and could not complete your request.
Please try again in 30 seconds.
It's very strange, as it seems to work for a few minutes, and only after a while get broken - why???
Thanks