IntroConformal: Conformal Factuality Guarantees for Large Vision-Language Models via Introspective Signals

arXiv:2609.01375v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have achieved strong multimodal performance, yet ensuring the factual correctness of generated content remains challenging. Existing methods that provide statistical guarantees on factuality typically rely on external verifiers or generation-time confidence signals, which introduce auxiliary dependencies or often fail for confident but incorrect outputs. We argue that reliable factuality control can instead be achieved through introspective signals derived from the model itself. We introduce IntroConformal, a training-free Conformal Risk Control (CRC) framework that provides finite-sample, distribution-free factuality guarantees. We first instantiate it with layer-wise semantic stability, a conformity score derived from hidden-state representations, and then propose verification probability, a stronger score capturing the model's self-administered judgment on claim factuality. Across multiple LVLM architectures, IntroConformal satisfies the conformal risk guarantee while substantially reducing abstention and achieving competitive or superior claim-level discrimination relative to external verifier-based baselines.

Related

• Mitigating Multimodal Hallucination via Phase-wise Self-reward
• When Relations Break: Analyzing Relation Hallucination in Vision-Language Model Under Rotation and Noise
• Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models
• HaluNet: Learning Hallucination Risk from Internal Signals in LLM Question Answering

Source: arXiv cs.CL | 2026-09-02