Kara: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression
DGX agentarXiv:2607.01237v1 Announce Type: cross Abstract: Reasoning language models often generate long chain-of-thought (CoT), which accumulates a massive KV cache during the decoding phase and incurs high d