Tutorials
Exploring High-Order Self-Similarity for Video Understanding
arXiv:2604.20760v1 Announce Type: new Abstract: Space-time self-similarity (STSS), which captures visual correspondences across frames, provides an effective way to represent temporal dynamics for vid
arXiv:2604.20760v1 Announce Type: new Abstract: Space-time self-similarity (STSS), which captures visual correspondences across frames, provides an effective way to represent temporal dynamics for video understanding. In this work, we explore higher-order STSS and demonstrate how STSSs at different orders reveal distinct aspects of these dynamics. We then introduce the Multi-Order Self-Similarity (MOSS) module, a lightweight neural module designed to learn and integrate multi-order STSS features. It can be applied to diverse video tasks to enhance motion modeling capabilities while consuming only marginal computational cost and memory usage. Extensive experiments on video action recognition, motion-centric video VQA, and real-world robotic tasks consistently demonstrate substantial improvements, validating the broad applicability of MOSS as a general temporal modeling module. The source code and checkpoints will be publicly available.
Related
- Chain-of-Glimpse: Search-Guided Progressive Object-Grounded Reasoning for Video Understanding
- Learning Long-term Motion Embeddings for Efficient Kinematics Generation
- FlowC2S: Flowing from Current to Succeeding Frames for Fast and Memory-Efficient Video Continuation
- Memorize When Needed: Decoupled Memory Control for Spatially Consistent Long-Horizon Video Generation
Source: arXiv cs.CV | 2026-04-23