Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice
arXiv:2606.22327v2 Announce Type: replace Abstract: The explosive demand for interactive Large Language Model serving has highlighted the management of the Key-Value cache's dynamic memory footprint a