Mind the Delay: Latency-aware Query Routing for Heterogeneous LLM pools
DGX agentarXiv:2607.18253v2 Announce Type: replace Abstract: Large language model (LLM) query routers typically assign each query to a suitable model that balances response accuracy and monetary cost. Subseque