Local Ai
LoopREG: Representation Supervision Enables Test-Time Recurrent Depth in Diffusion Transformers
arXiv:2610.17877v1 Announce Type: new Abstract: Reusing Transformer blocks offers a way to increase diffusion-model computation at test time, but naive recurrence can degrade generation beyond its tra
arXiv:2610.17877v1 Announce Type: new Abstract: Reusing Transformer blocks offers a way to increase diffusion-model computation at test time, but naive recurrence can degrade generation beyond its training depth. We introduce LoopREG, a representation-supervised recurrent diffusion Transformer that repeatedly applies a shared block at the state supervised by a pretrained visual encoder. Alignment is applied after the final recurrent visit during training; inference extends the same learned operator to additional visits. On ImageNet 256 x 256, LoopREG reduces SiT-B/2 FID from the local REG baseline's 15.08 to 4.91 at 400K training steps, a 67.41% reduction. Within a checkpoint trained with two visits, one extra inference visit improves FID from 14.71 to 5.34 with unchanged weights. The benefit persists on SiT-L/2 and SiT-XL/2. Controlled interventions show that useful recurrence follows the location of representation supervision, while REPA alignment produces a smaller extrapolation benefit. These results make representation-supervised recurrence a practical route to improving diffusion generation through additional inference depth.
Related
- Adaptive Depth in Looped Transformers: Diagnosing Learned Halting Gates and Trajectory Readouts
- AHPA: Adaptive Hierarchical Prior Alignment for Diffusion Transformers
- DynaTokens: Teaching Dynamics to Camera-Controlled Video Models at Test Time
- Memory-Efficient Training-Free Acceleration of Diffusion Transformers with BaryCache
Source: arXiv cs.CV | 2026-10-14