Do Vision-Language Models Understand Visual Persuasiveness? A Diagnosis via Visual Persuasive Factors

arXiv:2511.17036v2 Announce Type: replace Abstract: Visual persuasion uses images to shape cognition, emotion, and behavior, with its effects depending on both visual attributes and semantic context. Despite recent progress, it remains unclear whether Vision-Language Models (VLMs) understand visual persuasiveness. This motivates us to ask: can VLMs assess whether an image persuasively supports an intended message, which visual factors shape this judgment, and do they align with human judgments? Through empirical analyses on image-message pairs where human raters consistently agree on the persuasiveness judgment, we show that VLMs exhibit a recall-oriented bias: they over-predict images as persuasive while achieving high recall. We introduce Visual Persuasive Factors (VPFs), a taxonomy informed by cognitive psychology for quantifying visual cues that shape persuasive judgments. Our factor-level analysis reveals that VPFs distinguish human persuasiveness judgments, whereas VLMs only partially reproduce these patterns, often generating false positives by treating persuasion-relevant cues as sufficient evidence. Building on this insight, we evaluate VPF-guided interventions and find that properly framed VPF knowledge can improve performance, but merely specifying visual cues or adding step-by-step reasoning is insufficient. By analyzing model rationales at the level of functional reasoning steps, we further identify a central bottleneck in connecting object identification to semantic message alignment.

Related

• Is CLIP Cross-Eyed? Revealing and Mitigating Center Bias in the CLIP Family
• Real Images, Worse Judgments: Evaluating Vision-Language Models on Concreteness and Imagery
• Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications

Source: arXiv cs.CL | 2026-09-11