OmniKVQuant: KV Cache Quantization for Omni-LLMs
DGX agentarXiv:2609.11582v1 Announce Type: new Abstract: As Omni-modal large language models (Omni-LLMs) take in audio, video and text together, their KV cache memory cost grows. KV cache quantization is the d