Research
Putting Captions to the Test: Evaluating Video Caption Quality through Multiple-Choice Question Answering
Evaluating video captioning remains a critical challenge for Visual Large Language Models (VLLMs). Existing metrics primarily rely on matching generated text against ground-truth references. This para
Evaluating video captioning remains a critical challenge for Visual Large Language Models (VLLMs). Existing metrics primarily rely on matching generated text against ground-truth references. This paradigm suffers from the “one-to-many” nature of video description, where high-quality captions are often penalized for lexical mismatches or valid shifts in visual focus. Furthermore, such assessments are typically one-dimensional, failing to provide a fine-grained analysis of caption quality. To address this, we redefine caption quality via information fidelity: A caption must maximize the coverage…
Related
- Meaning in Order, Order in Meaning: Semantic R-precision for Keyphrase Evaluation
- MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering
- Test-Time Reasoners Are Strategic Multiple-Choice Test-Takers
Source: Apple ML Research | 2026-09-11