RADC: Risk-Aware Dual Caching for Vision-Language Test-Time Adaptation

arXiv:2610.06932v1 Announce Type: new Abstract: Cache-based test-time adaptation (TTA) for vision-language models is often hindered by background bias in global representations and unreliable entropy-based cache admission under representation variations. To address these limitations, we propose RADC, which enhances prototype learning through reliable dual caching. RADC introduces a Semantic Foreground Cache that aggregates category-consistent spatial evidence from CLIP representations, yielding foreground prototypes that complement the global cache while mitigating background interference. To reliably manage both caches, Gaussian Risk Admission models multi-view representations as diagonal Gaussian distributions and jointly considers class separation and feature uncertainty to prioritize reliable cache candidates. RADC integrates zero-shot logits with complementary global- and foreground-cache predictions for robust inference. Extensive experiments on cross-domain and out-of-distribution benchmarks demonstrate consistent state-of-the-art performance.

Related

• Position Aware Layer Queries for Test Time Training in Vision Language Models
• Ramen: Robust Test-Time Adaptation of Vision-Language Models with Active Sample Selection
• Test-Time Adaptation via Dual Distillation for Videos Under Severe Distribution Shifts
• CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Models

Source: arXiv cs.CV | 2026-10-07