When scaling a pod, will kubernetes start new pods on more available nodes?

Question

I tried to find a clear answer to this, and I'm sure it's in the kubernetes documentation somewhere but I couldn't find it.

If I have 4 nodes in my cluster, and am running a compute intensive pod which presumably uses up all/most of the resources on a single node, and I scale that pod to 4 replicas- will kubernetes put those new pods on the unused nodes automatically?

Or, in the case where that compute intensive pod is not actually crunching numbers at the time but I scale the pod to 4 replicas (ie: node running the original pod has plenty of 'available' resources) will kubernetes still see that there are 3 other totally free nodes and start the pods on them?

Answer 1

As far as I remember, it depends on the scheduler configuration. If you are running your on-premise kubernetes cluster and you have access to the scheduler process, you can tune the policy as you prefer (see documentation).

Otherwise, you can only play with the pod resource requests/limits and anti-affinity (see here).

Answer 2

Kubernetes will schedule the new pods on any node with sufficient resources. So one of the newly created pods could end up on a node were one is already running, if the node has enough resources left.

You can use an anti affinity to prevent pods from scheduling on the same node as a pod from the same deployment, e.g. by using the development's label

podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: resources-group
              operator: In
              values:
              - high
          topologyKey: topology.kubernetes.io/zone

Read docs for more on that topic.