Capacity Scheduling for Elastic Resource Sharing in Kubernetes - Qingcan Wang & Yuan Chen

Capacity Scheduling for Elastic Resource Sharing in Kubernetes - Qingcan Wang, Alibaba & Yuan Chen, Apple

Kubernetes manages resources capacity across multi-tenants/users/namespaces by allocating a fixed amount of resource quotas to each namespace. It lacks sufficient support of dynamic resource sharing within and across teams and organizations and can result in low cluster utilization. It has become a roadblock to migrating applications from other cluster management platforms (e.g., YARN) to Kubernetes. Qingcan Wang from Alibaba and Yuan Chen from Apple will present their collaborative work on a Kubernetes enhancement to address the issue. Capacity scheduling offers a similar feature to YARN’s capacity scheduler and enables elastic resource sharing to improve cluster utilization in Kubernetes. It supports hierarchical resource groups with guaranteed and maximum resources for dynamic sharing of resources, from CPU, memory, disk to extended resources like GPU. It is seamlessly integrated into Kubernetes as plugins and has been used in large scale production clusters such as Alibaba Cloud.

[ronaldpetty]

Very informative and well done Yuan and Qingcan.