Investigating Query-Insensitive Behavior in Spatio-Temporal Video Grounding

arXiv:2610.06018v1 Announce Type: new Abstract: Spatio-temporal video grounding (STVG) aims to localize objects or events described by natural language queries in both space and time. Existing STVG models are typically trained and evaluated under the assumption that each query is relevant to the input video. In this work, we challenge this assumption by studying the behavior of state-of-the-art STVG models under irrelevant queries and missing textual input. Our experiments show that current models can still produce plausible spatio-temporal predictions even when the query is unrelated to the video or removed entirely. We further analyze HCSTVG-v2 and VidSTG to identify dataset regularities that may encourage such query-insensitive behavior. Our study highlights an underexplored limitation of STVG models and motivates negative-aware evaluation protocols and architectures that explicitly assess query relevance.

Bridging Time and Space: Decoupled Spatio-Temporal Alignment for Video Grounding
Improving Spatial-Temporal Reasoning in Video-Language Models with Structured Video Prompting
Event-Driven Refresh and Recurrence Memory to Reduce Stale Grounding in Referring Video Object Segmentation
QCA: Query- and Content-Aware Keyframe Selection for Long Video Understanding

Source: arXiv cs.CV | 2026-10-06

Related