PixelDiT: Pixel Diffusion Transformers for Image Generation

arXiv:2511.20645v2 Announce Type: replace Abstract: Latent-space modeling has been the standard for Diffusion Transformers (DiTs). However, it relies on a two-stage pipeline where the pretrained autoencoder introduces lossy reconstruction, leading to error accumulation while hindering joint optimization. To address these issues, we propose PixelDiT, a single-stage, end-to-end model that eliminates the need for the autoencoder and learns the diffusion process directly in the pixel space. PixelDiT adopts a fully transformer-based architecture shaped by a dual-level design: a patch-level DiT that captures global semantics and a pixel-level DiT that refines texture details, enabling efficient training of a pixel-space diffusion model while preserving fine details. PixelDiT achieves 1.61 FID on ImageNet 256 and 1.81 FID on ImageNet 512, surpassing existing pixel generative models. We further extend PixelDiT to text-to-image generation and pretrain it at the 10242resolution in pixel space. It achieves 0.74 on GenEval and 83.5 on DPG-bench, approaching the best latent diffusion models. Code: https://github.com/NVlabs/PixelDiT

Related

• Latent Wavelet Diffusion For Ultra-High-Resolution Image Synthesis
• RectifiedHR: Enable Efficient High-Resolution Synthesis via Energy Rectification
• DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation
• An Analysis of Regularization and Fokker-Planck Residuals in Diffusion Models for Image Generation

Source: arXiv cs.CV | 2026-04-17