Research
Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes
AD2-Bench diagnoses multimodal reasoning via a Chain of Evidence, showing failures from spatial ambiguity and semantic uncertainty in adverse urban scenes.
Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes
What the paper does
Proposes AD2-Bench, a Hierarchical Visual Diagnosis framework that decomposes multimodal reasoning into a structured Chain of Evidence (CoE). Outcome-only benchmarks hide process failures; AD2-Bench targets complex urban scenes under adverse conditions where MLLMs often infer without sufficient visual evidence.
Key findings
- Reliability drops when models disconnect perception from reasoning in cluttered/adverse scenes.
- Two primary failure modes: Spatial Ambiguity (localization errors amid clutter) and Semantic Uncertainty (degraded features → wrong interpretation).
- Fine-grained CoE diagnosis shows robust multimodal reasoning depends on accurate evidence acquisition, not just final answers.
Why it matters
Shifts evaluation from scoreboards to diagnosable evidence chains—relevant for safety-critical vision-language agents and urban/robotics perception.
Related
Source: https://arxiv.org/abs/2506.09557 | 2025-06-11