TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation

arXiv:2610.15324v1 Announce Type: new Abstract: Autoregressive (AR) video diffusion models enable streaming generation of long videos but suffer from significant memory overhead due to the rapidly growing KV cache. Existing KV cache quantization methods are primarily designed for LLMs and transfer poorly to video models, where activations exhibit different statistics and strong spatiotemporal redundancy. We observe that the video KV cache is largely predictable over time, with a clear asymmetry. Keys are highly predictable from their temporal history, while values depend primarily on the current key and need a short history of preceding values. Based on these observations, we propose extbf{method}, a training-free predictive KV cache quantization framework for AR video generation. method predicts keys and values according to their distinct dependencies and stores only the residuals at low precision. Both predictors are lightweight, calibrated once offline by linear regression on a small set of BF16 samples, and remain fixed across prompts and generation runs. Experiments on Self-Forcing, Causal Forcing, and LongCat-Video-13B show that method consistently achieves higher generation quality than existing KV cache quantization methods at comparable or higher compression ratios, reaching up to 6.71imes KV cache compression with the best overall quality--efficiency tradeoff. Our code will be released on GitHub.

Source: arXiv cs.CV | 2026-10-14

Related