Running Large-Scale GPU Workloads on Kubernetes with Slurm

NVIDIA's open-source project Slinky (developed by SchedMD, now part of NVIDIA) enables organizations to run full Slurm clusters directly on Kubernetes infrastructure by managing the complete lifecycle of Slurm daemons as pods, solving the challenge of unifying Slurm's powerful HPC scheduling capabilities with Kubernetes' cloud-native GPU management platform. The slurm-operator component represents each Slurm element as Kubernetes Custom Resource Definitions, bidirectionally synchronizes node state between the two systems, and supports autoscaling with workload-safe drain-before-terminate behavior. NVIDIA runs Slinky in production across clusters scaling to over 8,000 GPUs for large-scale LLM training and multi-node inference, with GPU communication benchmarks (NCCL all-reduce and all-gather) showing no measurable performance overhead from the Kubernetes layer.

NestPipe: Large-Scale Recommendation Training on 1,500+ Accelerators via Nested Pipelining
Foundry: Template-Based CUDA Graph Context Materialization for Fast LLM Serving Cold Start
Measurement of Generative AI Workload Power Profiles for Whole-Facility Data Center Infrastructure Planning
RLBoost: Harvesting Preemptible Resources for Cost-Efficient Reinforcement Learning on LLMs
Cut Checkpoint Costs with About 30 Lines of Python and NVIDIA nvCOMP

Source: hardware

Related