ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios
DGX agentarXiv:2604.09603v1 Announce Type: cross Abstract: Speculative Decoding promises to accelerate the inference of Large Language Models, yet its efficacy often degrades in production-grade serving. Exist