GCloud K8s Pod-to-Pod Routing

1
votes

Currently, I'm running a Kubernetes cluster on GCloud K8s Engine.

Now I'm running an OpenVPN Server on there to create a network where multiple clients can talk together in a client-to-client fashion. If I'm using just a single VPN server it also already works. The client can connect to the K8s Pod and communicate with other clients or even the server itself.

However now I want to make that a little bit more available and want to have at least two servers which means I create another VPN network, which is relatively simple, by using the same configuration I used in server1 (I just need to adjust the Subnet).

But the tricky part is how can I make it happen that both pods can correctly route the networks?

i.e. I have the VPN networks 172.40.0.0/16 (Pod 1) and 172.41.0.0/16 (Pod 2). Does K8s or GCloud have any way of announcing the VPN network so that the pods will correctly route requests from 172.40.0.0/16 to 172.41.0.0/16

(OpenVPN will have both routes pushed to the client, so either Pod 1 will be the gateway or Pod 2)

I wouldn't bother writing code so that I can correctly communicate with the pods i.e. if I create a GCloud Route with the POD IP as a gateway with the networks would that work?

1
kubernetesgcloud
Not sure I got your question right, but in general it not the best idea to communicate to pods directly in k8s since pod can be moved/destroyed in anytime. Instead you can talk to services so you will be able to communicate with service by using its name thanks to KubeDNS. In your case even if you will set up new network, you can still communicate to its services just by their names – Evgeny Makarov
well the problem is that I can't have a service that can "route" a whole network, also if a pod is destroyed/moved (it's a statefulset) i will could just delete the old route and announce a new one. Basically I'm looking for an idea how I could solve this problem so if there is a way to have a service which will make it work that my vpn is working, everything would be ok. I mean I could also use the vpn to actually connect the vpn to the vpn itself (which would have an overhead) – Christian Schmitt
i.e. a cool way would be to have a VPC which the OpenVPN uses and the gcloud compute will actually do the routing automatically. – Christian Schmitt

1 Answers

1
votes

Does K8s or GCloud have any way of announcing the VPN network so that the pods will correctly route requests from 172.40.0.0/16 to 172.41.0.0/16

Kubernetes doesn't have any such mechanisms. However, you could look at BGP peering with Calico as an overlay.

The other option I guess is to create manual routes on both servers that point to each other that way traffic flows both ways. Traffic to the PodCidr is gonna be trickier because it's generally masqueraded with iptables and in a Kubernetes cluster the PodCidr is cluster-wide.