ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference
arXiv:2609.17943v1 Announce Type: cross Abstract: Long-context LLM inference is bottlenecked by attention, whose repeated KV-cache reads make decoding memory-bound. Self-speculative decoding alleviate