RGBX-Next: Towards Realistic Generative Rendering from G-Buffers

arXiv:2608.13929v1 Announce Type: new Abstract: Diffusion models have achieved impressive results in image, video, and streaming generation. However, compared to traditional 3D rendering, they still lack precise control over the generated output. We believe a viable path forward is to use generative models as learned renderers conditioned on traditionally rendered G-buffers. We introduce RGBX-Next, a unified generative framework for forward and inverse rendering, which allows estimating G-buffers from images, videos, and streams, and rendering realistic images, videos, and streams from G-buffers. Our key contribution is a general recipe for finetuning diffusion transformer (DiT) models into generative forward and inverse renderers. We show that the resulting models achieve high quality in both realistic generative rendering and intrinsic decomposition. We will make all our models publicly available. We believe that the design principles presented in this paper will benefit future research on controllable generative forward and inverse rendering.

Related

• RL-RIG: A Generative Spatial Reasoner via Intrinsic Reflection
• Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation
• Compositional Video Generation via Inference-Time Guidance

Source: arXiv cs.CV | 2026-08-17