SpatialVAM:Spatial-Aware Multi-View Video Diffusion as a Data-Efficient Robot Policy

arXiv:2604.03181v2 Announce Type: replace-cross Abstract: Robotic manipulation requires understanding both the 3D spatial structure of the environment and its temporal evolution, yet most existing policies neglect one or both aspects. They often rely on 2D visual observations or backbones pretrained on static image--text pairs, which leads to high data requirements and limited comprehension of environment dynamics. To address this, we introduce SpatialVAM, the first 3D Video Action Model that simultaneously predict spatial-aware multi-view heatmap videos and RGB videos. Our key insight is that this design naturally injects 3D information into video foundation models while aligning the representation format between video pretraining and action finetuning. Extensive experiments demonstrate that SpatialVAM enables data-efficient, robust, generalizable, and interpretable manipulation. With only ten demonstration trajectories and no additional pretraining, SpatialVAM handles challenging long-horizon and contact-rich tasks, generalizes to out-of-distribution settings, and predicts realistic future videos. Evaluations on Meta-World (22%parrow), RoboCasa (15%parrow) and real-world robotic platforms (16%parrow) show that SpatialVAM consistently outperforms other video action models, vision language action models and 3D-based policies, establishing a new state-of-the-art in data-efficient multi-task manipulation.

Related

• AnchorDream: Repurposing Video Diffusion for Embodiment-Aware Robot Data Synthesis
• DeVA: Decoupled Video-Action Model with physical guidance for robot policy learning
• Multi-View Relational Distillation for Spatial Reasoning with Vision-Language Models
• MV-Forcing: Long Multi-View Video Generation via 4D-Grounded Spatio-Temporal Self-Forcing

Source: arXiv cs.CV | 2026-08-14