Evaluating Vision-Language Models for Image Quality Assessment using Psychophysical Data

arXiv:2603.24578v2 Announce Type: replace Abstract: Psychophysical experiments remain the most reliable approach for perceptual image quality assessment (IQA), yet their cost and limited scalability motivate automated alternatives. This paper investigates whether Vision-Language Models (VLMs) can assist in assessing perceived image appearance and quality. We introduce a psychophysics-inspired framework to probe VLM perceptual sensitivity through controlled pairwise image comparisons of contrast, colorfulness, and overall preference. Six VLMs (four proprietary and two open-weight models) are compared against psychophysical data. Results reveal strong attribute-dependent variability: Claude exhibits the highest internal consistency, whereas GPT achieves the strongest agreement for overall preference. Claude and Qwen show the strongest alignment for colorfulness, while Qwen performs best for contrast. However, no model consistently matches human perception across all attributes. High self-consistency does not necessarily imply perceptual validity, and VLM--human agreement generally improves when perceptual differences among renderings are more pronounced.

Related

• Employing Vision-Language Models for Face Image Quality Assessment
• Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation
• What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective

Source: arXiv cs.CV | 2026-09-09