GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation

arXiv:2608.25659v1 Announce Type: new Abstract: Vision-Language-Action (VLA) policies have advanced language-conditioned robotic manipulation, yet action-imitation objectives provide only weak supervision for metric 3D structure and short-horizon physical evolution. Geometry-enhanced policies mainly improve current-scene grounding, whereas predictive policies often model future dynamics in RGB or latent spaces and may incur substantial deployment cost. GaussianDream demonstrates that training-time current Gaussian reconstruction and future Gaussian prediction provide effective 3D supervision, but its dense VGGT/TGE-based prefix jointly carries state, dynamics, and action-conditioning information. We present extbf{methodname}, a compact, policy-native extension that inserts extbf{World State Tokens} and extbf{World Prediction Tokens} directly into the VLA backbone. A training-only extbf{World Representation Head} decodes these tokens into a Current World and coupled Future Prediction over shared Gaussian primitives, while static--dynamic factorization preserves persistent structure and focuses residual motion on interaction-relevant regions. At inference, the head, renderer, auxiliary objectives, and VGGT/TGE pathway are removed, leaving only 20 world tokens without online Gaussian decoding or rollout. method achieves extbf{98.6%} on LIBERO and extbf{87.8%} on LIBERO-Plus, with clear gains under Camera and Layout shifts. Real-robot experiments further improve average success from 29.2% to 52.5% over reproduced pi_{0.5} while maintaining efficient closed-loop control.

Source: arXiv cs.RO | 2026-08-27

Related