autoscaling in multi tenant kubernetes cluster

Question

I am setting up kubernetes cluster on EC2, I have following requirements.

1. Support multiple tenants with predefined upper limits(resources utilization).
2. Scale up and down cluster based on current usage.

I know point 1 can be achieved by resource-quota, resource-quotas page also talks about scaling up cluster by writing custom controller. Can I simply use it for scaling down as well? Is there any open source implementation available?

Kubernetes provides cluster-autoscaler for autoscaling cluster, does it work when I have multiple namespaces? How does scale down impact resource quota of namespace?

What happens if total available resources are less than sum of resource quota of all namesapces?

Answer 1

Cluster Autoscaler makes sure there is enough space for all pods in the cluster, regardless of namespace they run in. Basically if there is a pod that can't be scheduled because there are not enough resources, CA will add more nodes. If there are underutilized nodes they may be removed by CA. It's orthogonal to quotas.

More specifically: quotas are a mechanism limiting how much 'stuff' can be created in namespace. They don't check if there are actual resources in the cluster - just enforce a per-namespace limit. If you go over limit you won't be able to create pod. If you're under the limit you can create it, regardless if there is a space for it in cluster or not. If there is no space and you have autoscaler running it will add more nodes to make space. If there is no space and you don't have autoscaler your pod will just stay 'pending' until something else is deleted.

Actual cluster size (and scaling cluster up or down) has no impact on quotas whatsoever.