How to assign equal deployments across multiple nodes

1
votes

This seems like it should be simple enough, but I haven't been able to find an answer.

We have three labeled nodes:

thalia0 thalia1 thalia2

and a Deployment spec, named mm-sa.

I would simply like to set number of replicas of mm-sa to 48 and have the scheduler assign 16 pods to each node.

The closest I could find to this, was here, but it looks like this is not yet avaialble: Node affinity for even spread of pods across multiple availability zones. I also found this, why-kubernetes-scheduler-ignores-nodeaffinity, but it says that "According to the documentation, nodeAffinity must exist for each node that can be used for a scheduled pod and the node having the biggest weight sum is chosen."

This seems like such a simple use-case, but I cannot figure out how to achieve it without using a naïve approach, as in defining 3-different Deployments named mm-sa1, ... mm-sa3 and using a nodeSelector to assign each accordingly. (NB: I am currently using a nodeSelector to assign 16-replicas of mm-sa on a single node).

1
kuberneteskubernetes-pod
The scheduler by default should spread the pods across the nodes; do you have a specific requirement that it be exactly 1/3 on each node, or do you just not want everything concentrated on a single node? - David Maze
Each of the 3 nodes are configured identically with RAM and CPU, so the 16-pods per node is an optimal config determined through testing. Thus, the 1/3 on each node is an ideal. - horcle_buzz
What happens if you just set replicas: 48 and deploy it, without specifically trying to influence the placement? - David Maze
Good question. There are other nodes that we wouldn't want the pods to run on, but since we have the imagePullPolicy set to Never for the Deployment spec (all required docker images are on the specific nodes), I assume the scheduler would only assign the pods to the nodes with the particular images. - horcle_buzz

1 Answers

2
votes

Kubernetes will automatically spread the pods in a replication controller or service across nodes in a single-zone cluster. So @David Maze's solution should apply if there are no other nodes. However since you mentioned that you have other nodes that you do not want to schedule on, I would recommend using the nodeSelector. By using nodeSelector, as long as the node has the same key-value pair in the YAML, the pod will be able to schedule on the node.

You can read more about nodeSelector here: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector