AnyView: Synthesizing Any Novel View in Dynamic Scenes

arXiv:2601.16982v2 Announce Type: replace Abstract: Modern generative video models excel at producing convincing, high-quality outputs, but struggle to maintain multi-view and spatiotemporal consistency in highly dynamic real-world environments. In this work, we introduce extbf{AnyView}, a diffusion-based video generation framework for extit{dynamic view synthesis} with minimal inductive biases or geometric assumptions. We leverage multiple data sources with various levels of supervision, including monocular (2D), multi-view static (3D) and multi-view dynamic (4D) datasets, to train a generalist spatiotemporal implicit representation capable of producing zero-shot novel videos from arbitrary camera locations and trajectories. We evaluate AnyView on standard benchmarks, showing competitive results with the current state of the art, and propose extbf{AnyViewBench}, a challenging new benchmark tailored towards extit{extreme} dynamic view synthesis in diverse real-world scenarios. In this more dramatic setting, we find that most baselines drastically degrade in performance, as they require significant overlap between viewpoints, while AnyView maintains the ability to produce realistic, plausible, and spatiotemporally consistent videos when prompted from extit{any} viewpoint. Results, data, code, and models can be viewed at: https://tri-ml.github.io/AnyView/

Source: arXiv cs.CV | 2026-09-14

Related