GCE Http load balancer not reaching instances - UNHEALTHY

1
votes

I followed the instructions here to setup the load balancer:

https://cloud.google.com/nodejs/getting-started/run-on-compute-engine#multiple_instances

My instances work fine, they are accessible via inst.ance.ip.addr:8080 and send a 200 at the root

The load balancer shows:

Error: Server Error

The server encountered a temporary error and could not complete your request. Please try again in 30 seconds.

Also the health-check always returns UNHEALTHY:

$gcloud compute backend-services get-health api-service

shows:

backend: https://www.googleapis.com/compute/v1/projects/#####/zones/us-central1-f/instanceGroups/api-server status: healthStatus: - healthState: UNHEALTHY instance: https://www.googleapis.com/compute/v1/projects/#####/zones/us-central1-f/instances/api-server-jg2e port: 8080 - healthState: UNHEALTHY instance: https://www.googleapis.com/compute/v1/projects/#####/zones/us-central1-f/instances/api-server-taxd port: 8080 kind: compute#backendServiceGroupHealth

I repeated the setup process like twenty times, from creating the template to the firewall rules. No errors at all while doing so. I coud verify all in the ui of the google cloud console.

I'm going crazy.

1
node.jscloudload-balancing
Sandman, not a solution but we're seeing the same issues on our side today. The same rule that we built out months ago is suddenly showing unhealthy despite being able to hit the instance directly and it being up and operational. – djneely
How can we get Google's attention? Is there a forum? – abelabbesnabi
I opened a ticket with them and flagged this post in it as well. See if this link works. Reviewing other listings in their bugs section seems folks are seeing the same issue. code.google.com/p/google-compute-engine/issues/… – djneely
Thanks! I added a comment to it. Maybe they will look at it soon. – abelabbesnabi

1 Answers

2
votes

After submitting the ticket and working through their response providing a debugging blog entry.

https://cloudplatform.googleblog.com/2015/07/Debugging-Health-Checks-in-Load-Balancing-on-Google-Compute-Engine.html

Our issue wound up being with "Failure 2". After rebooting the server, the necessary routes were not there. However, after doing:

/etc/init.d/google-address-manager restart

the necessary routes were populated, health checks were good, and the services were then available via public access.

I grabbed the patches from

https://github.com/GoogleCloudPlatform/compute-image-packages/releases

and installed the startup-scripts and daemon packages.

Hope that helps.

Additional note: We were running Debian 8 fully patched at this point