Novel View Synthesis using DDIM Inversion

arXiv:2508.10688v3 Announce Type: replace Abstract: Single-image novel view synthesis requires viewpoint-dependent structural changes while retaining source-scene information and inferring unobserved content. We formulate this task as a camera-conditioned initialization problem for a frozen pretrained latent diffusion model. At an intermediate DDIM inversion timestep, we decompose the source representation into a structural mean and a source noise residual. A pose-conditioned translation U-Net (TUNet) learns to predict the target-view mean from the source mean. We then introduce a score-guided fusion strategy: the predicted mean and source noise residual form a proposal that is evaluated by the frozen denoiser, whose noise prediction is combined with the translated mean to initialize DDIM sampling. Only TUNet is trained, while the pretrained model synthesizes the final view without explicit reconstruction or rendering of a shared 3D scene. Experiments on MVImgNet and RealEstate10K demonstrate competitive performance. Beyond image-level metrics, Wasserstein comparisons show that the corrected initialization yields trajectory-curvature distributions closer to those of ground-truth target-initialized sampling than raw fusion. These results support structural translation and source-conditioned initialization as a practical approach to reusing frozen diffusion priors for novel view synthesis.

Related

• Stabilizing Camera-Controlled Novel View Synthesis at Inference Time
• GeoNVS: Geometry Grounded Video Diffusion for Novel View Synthesis
• ORCA: Occlusion-Aware Refinement and Completion for Novel View Synthesis
• One View Is Enough! Monocular Training for In-the-Wild Novel View Generation
• GenNVS: Geometry-enhanced Novel View Synthesis via Disentangled 3D Prior

Source: arXiv cs.CV | 2026-10-14