Model Releases
A VLM Answer Is Not an Anomaly Score: Rank Compression in Training-Free Video Anomaly Detection
arXiv:2608.21244v1 Announce Type: new Abstract: Vision-language models enable training-free video anomaly detection by answering questions about video segments. VAD benchmarks, however, require a scal
arXiv:2608.21244v1 Announce Type: new Abstract: Vision-language models enable training-free video anomaly detection by answering questions about video segments. VAD benchmarks, however, require a scalar anomaly score for each segment and evaluate the resulting ranking using the AUROC or AP. A VLM-based detector should therefore define an answer interface: the answer scale specifies the admissible answers, and the readout rule maps the model's output distribution to a score. Because this interface can change the evaluated ranking, it is part of the detector rather than a formatting detail. The generated readout uses only the most likely answer, whereas the probability readout uses the full distribution over admissible answers. Across four 7-8B VLMs, the probability readout outperforms the generated readout for every tested combination of answer scale, benchmark, and metric, with average gains ranging from 5 to 13 points across the four benchmark-metric pairs. The gap arises because the generated readout keeps only one answer value per segment, so segment with different answer distributions can receive the same score and lose their relative order. We call this loss of relative order generated-answer rank compression. Even when the answer scale allows 91 answers, the generated readout produces only 4-18 distinct scores, whereas the probability readout retains substantially finer score resolution. The advantage persists under every decoding strategy, prompt wording, and joint scoring-explanation prompt we test. The answer interface is therefore a consequential component of VLM-based VAD and should be explicitly specified and evaluated.
Related
- COPRA: Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detection
- Video Active Perception: Effective Inference-Time Long-Form Video Understanding with Vision-Language Models
- FindIt: A Format-Informed Visual Detection Benchmark for Generalist Multimodal LLMs
- VLMaxxing through FrameMogging Training-Free Anti-Recomputation for Video Vision-Language Models
- CRAFT: Compression via Recursive Adaptive Fusion of Video Tokens for Vision-Language Models
Source: arXiv cs.CV | 2026-08-24