Are Image Generators Zero-Shot Perceivers? A Rigorous Evaluation

arXiv:2609.07884v1 Announce Type: new Abstract: Recent work, such as Vision Banana, shows that lightweight instruction tuning can enable an image generator to achieve state-of-the-art performance across multiple visual perception tasks. Motivated by this perspective, we ask how far image generators can go on public visual perception benchmarks in a zero-shot setting. We introduce ProbeGen, a benchmark for zero-shot generative perception that casts monocular depth estimation, referring/reasoning segmentation, and object counting as conditional generation tasks specified through text prompts, and compares 20 models in total---including proprietary and open-weight image generators, specialist perception models, and MLLMs---across 11 published benchmarks. We observe that pretrained image generators show measurable zero-shot perceptual competence, but with a clear trade-off: specialist models remain stronger for in-distribution accuracy and efficiency, while generative models are often more robust under distribution shift and better at compositional semantic reasoning. We hope this study helps establish zero-shot generative perception as a meaningful research direction and provides a useful foundation for future work at the intersection of visual generation and understanding.

Related

• TAME: Test-Time Adversarial Prompt Tuning via Mixture-of-Experts for Vision-Language Models
• Density-Aware Translation of Spurious Correlations in Zero-Shot VLMs
• Wasserstein-Aligned Localisation for VLM-Based Distributional OOD Detection in Medical Imaging
• What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective

Source: arXiv cs.CV | 2026-09-09