TokenButler: Token Importance is Predictable
DGX agentarXiv:2503.07518v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) rely on the Key-Value (KV) Cache to store token history, enabling efficient decoding of tokens. As the KV-Cache g