Overview of SHROOM-Visions 2026: A Shared Task on Hallucination Detection in Large Vision-Language Models

arXiv:2608.25662v1 Announce Type: new Abstract: In 2026, we held the fourth iteration of the SHROOM Shared Task series: SHROOM-Visions (extbf{S}hared-task on extbf{H}allucinations and extbf{R}elated extbf{O}bservable extbf{O}vergeneration extbf{M}istakes in extbf{Vision} language modelextbf{s}), which is hosted at the UncertaiNLP Workshop co-located with EMNLP 2026. Following the success of the 2024 and 2025 tasks, this time we aim to tackle hallucinations through a model-agnostic detection task focused on large vision-language models. Building on the recently introduced SHEEP dataset, designed for long-term evaluation across model generations, the task invites participants to detect and classify fine-grained hallucination spans in image-conditioned text generation (VQA, image captioning, etc.). The evaluation uses a five-class taxonomy of hallucinations spanning four languages: Chinese, English, French, and Italian. The shared task generated strong interest in the NLP community worldwide, with 27 teams contributing 600+ system submissions. The best systems achieve average scores of 0.58 in character-level correlation, 0.46 in label-conditioned correlation, and 0.51 in intersection-over-union (IoU) across four languages, outperforming the baselines by 30-40 points.

Related

• Risk-aware Selective Prompting for Hallucination Mitigation in Large Vision-Language Models
• Mitigating Multimodal Hallucination via Phase-wise Self-reward
• EnsemHalDet: Robust VLM Hallucination Detection via Ensemble of Internal State Detectors

Source: arXiv cs.CL | 2026-08-27