Cloud Foundry on Bluemix: No network connection when starting new app with binary-buildpack

Question

First, I've finaly found out what the problem was but still, I decided to write this question+answer for others (because I spent 6 hours with this issue).

So, what's the problem...

I have a Cloud Foundry app (on public Bluemix) based on binary-buildpack. Two days ago, everything was OK. But not since yesterday. My app crashed (probably during restaging or something similar) and never started again. I tried to push the app again and still the same result. Really frustrating...

Something about the backend... There is a shell script in my instance that runs one binary application. Generaly, the application should connect to database server (also on public Bluemix).

The problem: Everytime I tried to start the app, it crashed immediately. This is what I found in logs: dial tcp: lookup databaseserverdomain.com on 0.0.0.0:53: server misbehaving.

There are a couple of similar problems on StackOverflow but no answer that would be helpful for me.

So, the error means that something went wrong with TCP connection. Ok, but what exactly? That's the question I'm going to answer myself...

Answer 1

Sounds like your binary isn't capable in properly handling connection problems. I would rather fix that part since I guess it will crash anyway when there is a connection issue.

Answer 2

The solution was actually simple...

I edited my shell script and add ping google.com -count 3 before launching the application to test if there is a stable network connection. This worked.

The application got 2 more seconds and it was enough for network/router/whatever to establish the connection.

Hmm.. It seems that there is something wrong with network routing on Cloud Foundry/Bluemix since yesterday.