Pods don't communicate between each other when on different nodes (EKS)

1
votes

we have an EKS cluster with 4 nodes and 10 microservices (at this moment) on it. We used to have 2 nodes and didn't see much problems, but since increasing to 4, things have "randomly" stopped working. I believe that the pod of a node can not talk to another node of the cluster.

We have randomly "service unavailable" when the pods are not on the same node. When on the same node, it's all good. We use http://service.namespace:port in appsettings.

I thought Kubernetes would resolve this automatically? We have an EKS v1.12. Thanks

PS: We tried to "telnet IP port" and it worked from one of the pod. When we do http://service.namespace:port, it doesn't resolve.

1
kubernetesamazon-eks
are the new nodes in the same subnet of the old ones? - Efrat Levitan
try service.namespace.svc.cluster.local - Tummala Dhanvi
@TummalaDhanvi I just tried and we get this error: "error": "An invalid request URI was provided. The request URI must either be an absolute URI or BaseAddress must be set." The devs are using c# - shrimpy
I am assuming that you are using coredns in EKS, could you please add the output of kubectl get configmap coredns -n kube-system -o yaml - Tummala Dhanvi
could you please use just http://service:port if both the services are in the same namespace - Tummala Dhanvi

1 Answers

2
votes

If the services are in the same namespace you can use http://servicename:port

If they are in different namespaces then you can use FQDN http://servicename.namespace.svc.cluster.local:port