YOCO++: Enhancing YOCO with KV Residual Connections for Efficient LLM Inference
DGX agentarXiv:2604.13556v1 Announce Type: new Abstract: Cross-layer key-value (KV) compression has been found to be effective in efficient inference of large language models (LLMs). Although they reduce the m