UT-ACA: Uncertainty-Triggered Adaptive Context Allocation for Long-Context Inference

arXiv:2603.18446v2 Announce Type: replace Abstract: Long-context inference remains challenging for large language models due to attention dilution and out-of-distribution degradation. Context selection mitigates this limitation by attending to a subset of key-value cache entries, yet most methods allocate a fixed context budget throughout decoding despite highly non-uniform token-level contextual demands. To address this issue, we propose Uncertainty-Triggered Adaptive Context Allocation (UT-ACA), an inference-time framework that dynamically adjusts the context window based on token-wise uncertainty. UT-ACA learns an uncertainty detector that combines semantic embeddings with logit-based confidence while accounting for uncertainty accumulation across decoding steps. When insufficient evidence is indicated, UT-ACA selectively rolls back, expands the context window, and regenerates the token with additional support. Experiments show that UT-ACA substantially reduces average context usage while preserving generation quality in long-context settings. Code and dataset are available at https://github.com/Tommy307/UT-ACA.

Related

• AdaThink-Med: Optimizing Inference-Time Compute for Medical Reasoning via Uncertainty Quantification
• TIME: Temporally Intelligent Meta-reasoning Engine for Context-Triggered Explicit Reasoning
• Uncertainty-gated selection for block-sparse attention

Source: arXiv cs.CL | 2026-08-12