Tutorials
Zero-Shot Novel Depth Synthesis Using 3D Foundation Models Scene Representations
arXiv:2609.04174v1 Announce Type: new Abstract: 3D Foundation Models (3DFMs) such as VGGT have recently pushed the boundaries of 3D vision by predicting rich unified representations with feed-foward t
arXiv:2609.04174v1 Announce Type: new Abstract: 3D Foundation Models (3DFMs) such as VGGT have recently pushed the boundaries of 3D vision by predicting rich unified representations with feed-foward transformers. The scene representations learned by these models enable strong performance on multiple 3D vision tasks. In this paper, we investigate using their internal representations to infer 3D in the scene from new views. Our hypothesis is that in order to solve the task of 3D reconstruction, these models need to learn a representation that includes a large amount of general knowledge about 3D scenes. After showing that it is possible to decode hidden surfaces from internal 3DFM representations, we propose a method, Z3D, that estimates pointmaps in unseen views by doing latent diffusion on 3DFM representation. We show that Z3D can predict realistic depth maps for new views across multiple datasets.
Related
- Beyond Templates: Revisiting Zero-Shot Remote Sensing through Meta-Prompting
- Stream3Dv2: Geometric-Semantic Fusion Enhanced Streaming Zero-Shot 3D Scene Understanding
- Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence
Source: arXiv cs.CV | 2026-09-04