Inductive Visual Logic for Few-Shot Out-Of-Distribution Adaptation in VLMs

arXiv:2609.38362v1 Announce Type: new Abstract: Generative vision-language models (VLMs) such as Qwen-VL and LLaVA achieve strong zero-shot performance on tasks overlapping with their pretraining distribution, yet fail on specialized domains where the required discriminative features were never learned, a regime we term distant out-of-distribution (OOD). Standard adaptation methods cannot overcome this representational absence because they operate within the encoder's existing feature space. However, VLMs retain a robust descriptive capacity even when discrimination collapses: a model that cannot classify a medical scan can still articulate its visual patterns. Exploiting this asymmetry, we introduce Inductive Visual Logic (IVL), a training-free framework that constructs classification knowledge from the model's surviving descriptive ability. IVL extracts visual traits from few-shot support images through dual-mode prompting, combining semantic descriptions with primitive visual observations, and organizes them into per-class trait dictionaries. At inference, hierarchical filtering identifies spatially grounded trait evidence for classification. Across multiple distant-OOD benchmarks, IVL achieves the highest aggregate accuracy under two VLM backbones while producing interpretable, trait-traceable predictions.

Related

• DESA-TTA: Dynamic EMA and Source Anchoring for Test-Time Adaptation
• What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective
• MuRA: Multi-Rank Adaptation for Efficient and Effective Test-Time Vision-Language Generalization
• GATE: Reliability-Gated Gaussian Evidence Fusion for Training-Free Test-Time Adaptation of Vision-Language Models
• Majorization-Guided Test-Time Adaptation for Vision-Language Models under Modality-Specific Shift

Source: arXiv cs.CV | 2026-10-01