Can't connect to Postgres (installed through Kubernetes Helm) service from external machine, connection refused

3
votes

I just installed Kubernetes with minkube on my desktop(running Ubuntu 18.10) and was then trying to install Postgresql on the desktop machine using Helm.

After installing helm, I did:

helm install stable/postgresql

When this completed successfully, I forwarded postgres port with:

kubectl port-forward --namespace default svc/wise-beetle-postgresql 5432:5432 &

and then I tested connecting to it locally from my desktop with: psql --host 127.0.0.1 -U postgres which succeeds.

I attempted to connect to postgres from my laptop and that fails with:

psql -h $MY_DESKTOP_LAN_IP -p 5432 -U postgres

psql: could not connect to the server: Connection refused
Is the server running on host $MY_DESKTOP_LAN_IP and accepting TCP/IP connections on port 5432?

To ensure that my desktop was indeed listening on 5432, I did:

netstat -natp | grep 5432

(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp        0      0 127.0.0.1:5432          0.0.0.0:*               LISTEN               17993/kubectl       
tcp6       0      0 ::1:5432                :::*                    LISTEN      17993/kubectl

Any help anyone? I'm lost.

2
postgresqlkuberneteskubernetes-helm
So to debug further I just installed mysql via helm as well and that is giving me the same issue. I can't access the mysql from my laptop. So looks like I can't access any of my Kubernetes services from an external machine. Searching around this appears to be a known problem: github.com/kubernetes/kubernetes/issues/42413 but the solution is not clear. Not sure how to use the command: iptables -P FORWARD ACCEPT. Simply running that command on my kubernetes machine didn't fix anything. – J. Cool
Did you find an answer to this? I've hit the same problem in that I'm not sure how to modify the PGSQL configuration using the Helm chart values.yaml file. – Mike Stoddart
@MikeStoddart Are you using minikube? – J. Cool
No, kubectl + k3s + k3d. – Mike Stoddart
@MikeStoddart I've posted what solved the problem for me. Check out my solution that I've just posted. – J. Cool

2 Answers

1
votes

you need to configure postgresql.conf to allow external client connections look for listen parameter and set it to *, it is under your postgres data directory, and then add your laptop's ip in pg_hba.conf. It controls the client access to your postgresql server, more on this here - https://www.postgresql.org/docs/9.3/auth-pg-hba-conf.html

0
votes

In my case the solution was a little bit of deeper understanding of networking.

For clarity, let's call the machine on which minikube is installed "A". The IP of this machine as it is visible from other computers on my Wifi maybe be say: 192.100.200.300.1 Since Postgres was being exposed on port 5432, my expectation was that postgres should be visible externally on: 192.100.200.300.1:5432. But this understanding is wrong which is what was leading to unexpected behavior.

The problem was that minikube runs in a VM and it gets its own IP address. It doesn't simply use the IP of the machine on which it is running. Minikube's IP is different from the IP of the machine on which it is running. To find out the IP of minikube, run: minikube ip. Let's call this IP $MINIKUBE_IP.

And then I had to setup port forwarding like:

kubectl port-forward --address "192.100.200.300" --namespace default svc/wise-beetle-postgresql 5000:5432 &

So now, if you called a service on: 192.100.200.300:5000 it would be forwarded to port 5432 on the machine which is running minikube and then 5432 would be received by your postgres instance.

Hope this untangles or clarifies this problem that others might encounter.