Streaming4D: Accelerate 4D World Models via Block-wise Video Generation and Incremental Reconstruction

arXiv:2609.00610v1 Announce Type: new Abstract: Current 4D generation paradigms are often bottlenecked by a sequential decoupling design: video is generated first, followed by 3D reconstruction, leading to high interaction latency. This limits applications in interactive real-time scenarios. To this end, we propose extbf{Streaming4D}, a tightly coupled synchronous pipeline that integrates block-wise autoregressive video generation with incremental 3D reconstruction. Unlike traditional frame-by-frame emission and delayed geometry recovery, Streaming4D generates temporal video blocks and immediately triggers reconstruction for each completed block, enabling parallel execution between synthesis and geometric updates. This approach allows the world representation to evolve online with the video stream, reducing feedback latency while preserving geometric fidelity. We instantiate extbf{Streaming4D} using a Self-Forcing-style autoregressive generator and an incremental reconstruction backend. Experiments show consistent runtime improvements across resolutions on a single RTX 4090 (1.24imes speedup), while maintaining high-quality 4D geometry and multi-view consistency.

Related

• DensityKV: Density-Guided KV Cache Compression for Long Video Generation
• Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory
• Partition the Support, Reconstruct the Residual: Training-Free Sparse Attention for Video Generation and World Models
• AdaState: Self-Evolving Anchors for Streaming Video Generation

Source: arXiv cs.CV | 2026-09-02