CacheBridge: Efficient Cross-Model KV Cache Transfer

arXiv:2609.00891v1 Announce Type: new Abstract: Sharing context between LLMs in a multi-model system requires the receiving model to prefill the shared prefix because KV caches are model-specific. Recent closed-form cross-model KV transfer, hereafter Full-Head Mapping, avoids this replay by fitting a training-free affine mapper from source to target caches. However, its full-head design maps each target KV head from every source KV head in the selected layers, making transfer quality sensitive to architectural differences and causing mapper storage and application cost to grow with layer support. To this end, we introduce CacheBridge, which co-designs architecture-indexed mapper support, attention-aligned calibration, and bounded mapper construction while retaining a closed-form affine interface for online deployment. CacheBridge restricts each target head to a matched source head, weights reconstruction errors by causal attention sensitivity, and uses a fused GPU kernel to construct weighted sufficient statistics without materializing full observation tensors. Across three transfer directions, CacheBridge recovers the two Ministral 3 transfer directions where Full-Head Mapping loses substantial accuracy while preserving 99.83% mean target retention on Qwen3. On Qwen3 14Bo32B, it reduces mapper storage by 8imes, accelerates application by up to 3.0imes, matches fullhead with one tenth of the calibration data, and reduces 500-sequence construction from 92.63 to 8.63 seconds (10.7imes).

Related

• RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention
• Stream2LLM: Overlap Context Streaming and Prefill for Reduced Time-to-First-Token (TTFT)
• BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching

Source: arXiv cs.AI | 2026-09-02