Think, Look, and Revise: Inconsistency-Aware Visual Self-Correction in MLLMs

arXiv:2608.29374v1 Announce Type: new Abstract: Tool-augmented multimodal reasoning integrates external tools (e.g., object detection, depth estimation) into multimodal large language models (MLLMs) to address perceptual bottlenecks in complex visual tasks. However, existing approaches rarely verify tool outputs, limiting their ability to detect and recover from tool failures. We propose ReVISE, a framework that equips MLLMs with verification and dynamic error recovery for tool-augmented reasoning. ReVISE introduces (1) a curated training dataset that supervises reflective behaviors, enabling models to validate tool-derived evidence, reformulate queries when visual mismatches arise, and fall back to intrinsic grounding when external tools are unreliable; and (2) a reinforcement learning based targeted rewards that encourage internal reflection and penalize spatial misalignment. Experiments on several benchmarks demonstrate consistent improvements over existing methods, highlighting the importance of error detection and correction in tool-augmented multimodal reasoning.

Related

• Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains
• Deeper Thought, Weaker Aim: Understanding and Mitigating Perceptual Impairment during Reasoning in Multimodal Large Language Models
• Attention-guided Fine-tuning of Multimodal Large Language Models Improves Chain-of-Thought Reasoning
• When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs
• Mimic Human Cognition, Master Multi-Image Reasoning: A Meta-Action Framework for Enhanced Visual Understanding

Source: arXiv cs.CV | 2026-09-01