VASC: Value-Aware Sparse Attention with Cross-Layer Memory for Efficient 3D Reconstruction

arXiv:2610.01013v1 Announce Type: new Abstract: Feed-forward 3D vision models such as VGGT have achieved remarkable progress, unifying camera estimation and dense scene reconstruction in a single pass. However, their quadratic global attention makes long image sequences expensive, while existing sparse methods may favor highly attended yet value-redundant regions. To address these limitations, we introduce VASC, a training-free sparse attention method combining value-aware block selection and execution-aware cross-layer memory. Our value-aware block selection integrates pooled query--key relevance with neighboring value contrast, reducing redundancy while preserving query-relevant and distinctive content. Cross-layer memory tracks unserved demand across layers and updates this state according to actual execution, enabling previously underserved blocks to compete under a fixed computation budget. Experiments on 7Scenes and NeuralRGB-D with VGGT and pi^3 demonstrate improved pose estimation and reconstruction quality compared with FasterVGGT, together with up to 2.29imes faster inference than dense VGGT. Code is available at https://github.com/kosakayamahoo-design/VASC.

Related

• ReSS: Residual-Restoring Sparse Attention for 3D Vision Transformers
• Block-Sparse Global Attention for Efficient Multi-View Geometry Transformers
• VGGT-Prime: Compute-Adaptive Mixture-of-Heads for Efficient Visual Geometry Transformers
• Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction
• SAF3R: Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformers

Source: arXiv cs.CV | 2026-10-02