img_v3_02fr_9d75bfc9-bb9a-4eac-be99-a89d28334a6g.png

微信图片_20241031204349.png

中关村国家自主创新示范区-会议中心

中国开源年会 2024

李孟轩.png

With AI's growing popularity, Kubernetes has become the de facto AI infrastructure. However, the increasing number of clusters with diverse AI devices (e.g., NVIDIA, Intel, Huawei Ascend) presents a major challenge.
AI devices are expensive, how to better improve resource utilization? How to better integrate with K8s clusters? How to manage heterogeneous AI devices consistently, support flexible scheduling policies, and observability all bring many challenges
The HAMi project was born for this purpose. This session including:
* How K8s manages heterogeneous AI devices (unified scheduling, observability)
* How to improve device usage by GPU share
* How to ensure the QOS of high-priority tasks in GPU share stories
* Support flexible scheduling strategies for GPU (NUMA affinity/anti-affinity, binpack/spread etc)
* Integration with other projects (such as volcano, scheduler-plugin, etc.)
* Real-world case studies from production-level users.
* Some other challenges still faced and roadmap

HAMi is currently the only sandbox project in the CNCF community that focuses on this areas.

如何通过 HAMi 解锁 AI infra 面临的 AI 设备使用率与异构设备管理的两大挑战

https://kaiyuanshe.com/activity/COSCon-2024/agenda/recFqfTa9U

https://kaiyuanshe.com/activity/COSCon-2024/agenda/recFqfTa9U/invitation

Presentation如何通过 HAMi 解锁 AI infra 面临的 AI 设备使用率与异构设备管理的两大挑战