Jumping the Line: Exploiting Length Predictions in LLM Scheduling
DGX agentarXiv:2610.03430v1 Announce Type: new Abstract: Efficient request scheduling is increasingly important for reducing completion time in large language model (LLM) serving. Size-based policies such as S