Token Latency Fairness: Performance Isolation for Multi-Tenant LLM Serving
arXiv:2609.18112v1 Announce Type: cross Abstract: LLM serving is typically offered as a shared, multi-tenant service, where high-demand workloads from one client can cause latency SLO violations for o