Adaptive DHiSparse: Speculative Decoding over a Load-Adaptive KV Hierarchy
DGX agentarXiv:2610.13708v1 Announce Type: new Abstract: Long-context serving runs out of KV memory before the hardware reaches its useful throughput. HiSparse stores history in host memory and fetches only th