EgoArgus: Benchmarking VLMs as Situational Assistants for Modality-Grounded User Supports

arXiv:2608.25561v1 Announce Type: new Abstract: VLMs are increasingly positioned as daily assistants that perceive first-person environments, follow user dialogue, and decide how to help. Existing egocentric benchmarks mainly evaluate visual understanding in isolation, leaving open whether models can arbitrate between visual evidence and user-provided language when the two are helpful, irrelevant, or conflicting. We introduce EgoArgus, a human-annotated dataset for evaluating egocentric assistants on understanding and decision tasks in five dialogue-video daily scenarios. Our results demonstrate that it is still challenging for current VLMs as reliable egocentric assistants, which requires identifying which modality is trustworthy and deciding when intervention is warranted. Deeper analysis also shows that existing modality bias mitigation methods are quite restricted to enhance performance, providing insights to aid practioners into the deployment of current VLMs as daily assistants.

Related

• Beyond Local Surprise: Grounded Dialogue as Selective Belief Revision under Referential Uncertainty
• Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models
• When Safety Overrides Vision: Exploring Dynamics between Vision Influence and Safety Alignment in Vision-Language Models

Source: arXiv cs.CL | 2026-08-27