Organization of pods in cluster in Google Container Engine

Question

What is the usual way to organize pods in a cluster in Kubernetes?

I have a Jenkins build server, Docker registry, Git repository and other development tools that I want to run in Google Container Engine. Do I create one cluster for each of them? Or can multiple pods be scheduled on each node?

So my question is: Would you create one cluster that holds all these services, or multiple clusters? The same question applies to production, qa etc enviroments. Do I create one cluster for each enviroment or do I have them in the same cluster?

Answer 1

To answer your first question, multiple pods can be scheduled on each node.

One of the best parts about Google Container Engine / Kubernetes is that it is really flexible, so you can structure your services in the way that works best for you. For your specific use case, I think that a single cluster would make sense because all of the applications that you want to run are closely related. You'll want to think a bit about choosing an appropriate size for your cluster (both the number of VMs and the size of each VM) to fit your entire workload.

You can experiment with creating a single cluster for both your QA and Prod workloads, or you can split them across clusters. Until Kubernetes has better support for QoS (for scheduling pods), it probably makes more sense to keep the QA environment separate (and probably sized more modestly).

Answer 2

Usually the pods are hosted by different nodes on our cluster depending on the resources the pod would need and also that the running pod may not overload the node. Moreover a single node in a cluster can host multiple pods.

Specific to the question,when talking about all the development tools all the pods here should be hosted on a single node only because all of them have to communicate with each other . When the question arises for different environments it would be wise to host them on different nodes but same cluster.

Kubernetes provides us with the added advantage of selecting nodes on which we wish to run our pods. The NodeSelector concept can come handy in this case.

kubectl get nodes

kubectl get pods

Label any node that you have in your cluster :

kubectl label nodes  gke-cluster1-default-pool-4db7fabf-zzx9 disktype=ccd

kubectl get nodes --show-labels

Now create a file to create a pod and mention the nodeSelector in that file.

nano task6pod.yaml

apiVersion: v1
kind: Pod
metadata:
 name: pod6
 labels:
   env: test
spec:
  containers:
  - name: container6
    image: nginx
  nodeSelector:
   disktype: ccd

nodeSelector in this file is the same label as of the label of the node mentioned earlier.

kubectl create -f task6pod.yaml

kubectl get pods -o wide

After this command you can see that the newly created pod will have the node that you wanted the one with the label.