Tango: Taming Visual Signals for Efficient Video Large Language Models

arXiv:2604.09547v1 Announce Type: new Abstract: Token pruning has emerged as a mainstream approach for developing efficient Video Large Language Models (Video LLMs). This work revisits and advances the two predominant token-pruning paradigms: attention-based selection and similarity-based clustering. Our study reveals two critical limitations in existing methods: (1) conventional top-k selection strategies fail to fully account for the attention distribution, which is often spatially multi-modal and long-tailed in magnitude; and (2) direct similarity-based clustering frequently generates fragmented clusters, resulting in distorted representations after pooling. To address these bottlenecks, we propose Tango, a novel framework designed to optimize the utilization of visual signals. Tango integrates a diversity-driven strategy to enhance attention-based token selection, and introduces Spatio-temporal Rotary Position Embedding (ST-RoPE) to preserve geometric structure via locality priors. Comprehensive experiments across various Video LLMs and video understanding benchmarks demonstrate the effectiveness and generalizability of our approach. Notably, when retaining only 10% of the video tokens, Tango preserves 98.9% of the original performance on LLaVA-OV while delivering a 1.88x inference speedup.

Related

• Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models
• Token Reduction via Local and Global Contexts Optimization for Efficient Video Large Language Models
• Bridging Time and Space: Decoupled Spatio-Temporal Alignment for Video Grounding
• ActFER: Agentic Facial Expression Recognition via Active Tool-Augmented Visual Reasoning

Source: arXiv cs.CV | 2026-04-13