Competition Domains Matter: Selective Head Globalization for Training-Free Sparse Attention in Video Diffusion Transformers

arXiv:2610.14672v1 Announce Type: new Abstract: Training-free sparse attention is a practical route to efficient video diffusion transformer inference, yet existing methods typically fix the competition domain at each query row for every attention head. Prior work primarily addresses how to estimate token-level importance within this fixed domain; whether the domain itself should adapt to head heterogeneity remains under-specified. We introduce EG-Sparse, a training-free framework that routes low-entropy heads to a grouped cross-head pool while retaining per-row local selection for high-entropy heads, using only scores already materialized by the sparse pipeline. A controlled diagnostic fixes the ranking score and varies only the competition rule: grouped cross-head selection reduces low-entropy-head reconstruction error by 9.8%, whereas forcing high-entropy heads into an all-head global pool increases it by 7.9%. Across four configurations, EG-Sparse achieves higher video quality while reducing attention density and GPU denoising latency compared with strong training-free sparse-attention baselines. Mechanism ablations further show that selective domain adaptation outperforms uniform, random, and fixed domain-assignment policies. These results support competition-domain adaptation as a distinct design axis within centroid-guided training-free sparse attention.

Related

• SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention
• AdaCluster: Adaptive Query-Key Clustering for Sparse Attention in Video Generation
• iCATS: Fast Video Generation via Interaction-Aware Sparse Attention and Timestep-Adaptive Sparsity
• DFSAttn: Dynamic Fine-grained Sparse Attention for Efficient Video Generation

Source: arXiv cs.CV | 2026-10-14