Local Ai
CodeComp: Structural KV Cache Compression for Agentic Coding
arXiv:2604.10235v1 Announce Type: new Abstract: Agentic code tasks such as fault localization and patch generation require processing long codebases under tight memory constraints, where the Key-Value
arXiv:2604.10235v1 Announce Type: new Abstract: Agentic code tasks such as fault localization and patch generation require processing long codebases under tight memory constraints, where the Key-Value (KV) cache becomes the primary inference bottleneck. Existing compression methods rely exclusively on attention signals to estimate token importance, systematically discarding structurally critical tokens such as call sites, branch conditions, and assignments that are essential for code understanding. We present CodeComp, a training-free KV cache compression framework that incorporates static program analysis into LLM inference via Code Property Graph priors extracted by Joern. Across bug localization and code generation benchmarks, CodeComp consistently outperforms attention-only compression baselines under equal memory budgets, recovering the majority of full-context accuracy under aggressive KV cache compression, while matching the patch generation quality of uncompressed full-context inference and integrating seamlessly into SGLang-based agentic coding pipelines without model modification.
Related
- SepSeq: A Training-Free Framework for Long Numerical Sequence Processing in LLMs
- Prune-Quantize-Distill: An Ordered Pipeline for Efficient Neural Network Compression
- Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures
- QCFuse: Query-Centric Cache Fusion for Efficient RAG Inference
- Enabling Global, Human-Centered Explanations for LLMs:From Tokens to Interpretable Code and Test Generation
Source: arXiv cs.CL | 2026-04-14