Pod does not communicate with other pod through service

1
votes

I have 2 pods: a server pod and a client pod (basically the client hits port 8090 to interact with the server). I have created a service (which in turn creates an endpoint) but the client pod cannot reach that endpoint and therefore it crashes:

Error :Error in client :rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial tcp :8090: connect: connection refused")

The client pod tries to access port 8090 in its host network. What I am hoping to do is that whenever the client hits 8090 through the service it connects to the server.

I just cannot understand how I would connect these 2 pods and therefore require help.

server pod:

apiVersion: v1
kind: Pod
metadata:
  name: server-pod
  labels:
    app: grpc-app
spec:
  containers:
  - name: server-pod
    image: image
    ports:
      - containerPort: 8090

client pod :

apiVersion: v1
kind: Pod
metadata:
  name: client-pod
  labels:
    app: grpc-app
spec:
  hostNetwork: true
  containers:
  - name: client-pod
    image: image

Service:

apiVersion: v1
kind: Service
metadata:
  name: server
  labels:
    app: grpc-app
spec:
  type: ClusterIP
  ports:
  - port: 8090
    targetPort: 8090
    protocol: TCP
  selector:
    app: grpc-app
2
kubernetes
Why are both the server and client labelled as app: grpc-app? Won't that confuse the service as to where to send the connection? - Llama
@john Shouldn't they have same names to connect to the service ? (I'm really new to this so pardon me if I'm wrong ) - Tanmay Shrivastava
K8s uses the "selector" on the service to find which server pods provide the service. The client shouldn't have matching labels. You can connect to the service by name (server from the client pod) assuming the namespace is the same. If this solves your issue, let me know and I'll add an answer :) - Llama
@john how do I connect by service name ? Should I add it in the yml? - Tanmay Shrivastava

2 Answers

2
votes

Your service is selecting both the client and the server. You should change the labels so that the server should have something like app: grpc-server and the client should have app: grpc-client. The service selector should be app: grpc-server to expose the server pod. Then in your client app, connect to server:8090. You should remove hostNetwork: true.

2
votes

One thing that i feel is going wrong is that the services are not ready to accept connection and your client is trying to access that therefore getting a connection refused.I faced the similar problem few days back. What i did is added a readiness and liveness probe in the yaml config file.Kubernetes provides liveness and readiness probes that are used to check the health of your containers. These probes can check certain files in your containers, check a TCP socket, or make HTTP requests.

A sample like this

spec:
      containers:
      - name: imagename
        image: image
        ports:
        - containerPort: 19500
          name: http
        readinessProbe:
          httpGet:
            path: /health
            port: http
          initialDelaySeconds: 120
          periodSeconds: 5
        livenessProbe:
          httpGet:
            path: /health
            port: http
            scheme: HTTP
          initialDelaySeconds: 120
          timeoutSeconds: 5

So it will check whether your application is ready to accept connection before redirecting traffic.