LLM Serving Optimization with Variable Prefill and Decode Lengths
DGX agentarXiv:2508.06133v4 Announce Type: replace-cross Abstract: We study offline scheduling for large language model (LLM) serving under a fixed KV-cache memory budget, where requests have heterogeneous pro