Physically Partitioned KVCache Format for CPU--GPU Load Balancing in MoE Inference
DGX agentarXiv:2609.14507v1 Announce Type: cross Abstract: Single-GPU long-context inference with Mixture-of-Experts (MoE) models requires spilling the key-value cache (KVCache) to CPU memory. The spilled KV s