Model Releases
SOVABench: A Vehicle Surveillance Action Retrieval Benchmark for Multimodal Large Language Models
arXiv:2601.04824v3 Announce Type: replace Abstract: Automatic identification of events and recurrent behavior analysis are critical for video surveillance. However, most existing content-based video r
arXiv:2601.04824v3 Announce Type: replace Abstract: Automatic identification of events and recurrent behavior analysis are critical for video surveillance. However, most existing content-based video retrieval benchmarks focus on scene-level similarity and do not evaluate the action discrimination required in surveillance. To address this gap, we introduce SOVABench (Surveillance Opposite Vehicle Actions Benchmark), a real-world retrieval benchmark built from surveillance footage and centered on vehicle-related actions. SOVABench defines two evaluation protocols (inter-pair and intra-pair) to assess cross-action discrimination and temporal direction understanding. Although action distinctions are generally intuitive for human observers, our experiments show that they remain challenging for state-of-the-art vision and multimodal models. Leveraging the visual reasoning and instruction-following capabilities of Multimodal Large Language Models (MLLMs), we present a training-free framework for producing interpretable embeddings from MLLM-generated descriptions for both images and videos. The framework achieves strong performance on SOVABench as well as on several spatial and counting benchmarks where contrastive Vision-Language Models often fail. The code, annotations, and instructions to construct the benchmark are publicly available.
Related
- Towards Effective Long Video Understanding of Multimodal Large Language Models via One-shot Clip Retrieval
- VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification
- TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs
Source: arXiv cs.CV | 2026-10-14