Model Releases
CoMMET: A Psychologically Grounded Benchmark for Evaluating Theory of Mind in Multimodal LLMs
arXiv:2603.11915v2 Announce Type: replace Abstract: Theory of Mind (ToM)-the ability to reason about the mental states of oneself and others-is a cornerstone of human social intelligence. As Multimoda
arXiv:2603.11915v2 Announce Type: replace Abstract: Theory of Mind (ToM)-the ability to reason about the mental states of oneself and others-is a cornerstone of human social intelligence. As Multimodal Large Language Models (MLLMs) become ubiquitous in real-world applications, validating their capacity for this level of social reasoning is essential for effective and natural interactions. However, existing benchmarks for assessing ToM in MLLMs are limited; most rely solely on text inputs and focus narrowly on belief-related tasks. In this paper, we propose a new multimodal benchmark dataset, CoMMET, a comprehensive mental states and moral evaluation task inspired by the Theory of Mind Booklet Task. CoMMET expands the scope of evaluation by covering a broader range of mental states and introducing multi-turn testing. To the best of our knowledge, this is the first psychology-grounded benchmark to evaluate MLLMs across multiple mental states in a multimodal, open-ended, and multi-turn setting. Through a comprehensive assessment of MLLMs across different families and sizes, we analyze the strengths and limitations of current models and identify directions for future improvement.
Related
- Infusing Theory of Mind into Socially Intelligent LLM Agents
- Zing: Social Mind for LLMs
- MeetingToM: Evaluating Multimodal LLMs on Theory-of-Mind Reasoning in Multi-Party Meetings
- Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mind
Source: arXiv cs.CL | 2026-09-02