OUTLETS: Output-Length Prediction from Speculative Decoding Backbones
DGX agentarXiv:2609.01068v1 Announce Type: new Abstract: The heavy-tailed distribution of output lengths in Large Language Model (LLM) serving poses major challenges for resource provisioning and cluster sched