Research
Which one is banana man? Evaluating vision-language models in multi-turn pragmatic interpretation
arXiv:2608.29571v1 Announce Type: new Abstract: Flexible adaptation to context and shared pragmatic intuitions contribute to smooth human conversation. Iterated reference games---in which players repe
arXiv:2608.29571v1 Announce Type: new Abstract: Flexible adaptation to context and shared pragmatic intuitions contribute to smooth human conversation. Iterated reference games---in which players repeatedly pick out novel referents using language---present a test case for agents' ability to perform context-sensitive pragmatic reasoning in multi-turn linguistic environments. We tested humans and vision--language models on their ability to identify the intended meaning of descriptions produced in iterated reference games, varying the provided context in terms of amount, order, and relevance. While humans performed well consistently, the models we evaluated could make use of prior context to interpret humans' referring expressions, but they struggled to build up the relevant context to interpret those expressions effectively. Our results suggest that the models we evaluated lack core skills needed for efficient linguistic collaboration.
Related
- Evaluating Pragmatic Reasoning in Large Language Models: Evidence from Scalar Diversity
- Arbitration Failure, Not Perceptual Blindness: How Vision-Language Models Resolve Visual-Linguistic Conflicts
- Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding
Source: arXiv cs.CL | 2026-09-01