Tutorials
Shared Is Not Enough: Predictor Width and Factor Readability Under Masked Latent Prediction
arXiv:2610.32562v1 Announce Type: cross Abstract: Masked latent prediction methods such as I-JEPA aim to learn predictable features, but what determines which information they retain is less clear. We
arXiv:2610.32562v1 Announce Type: cross Abstract: Masked latent prediction methods such as I-JEPA aim to learn predictable features, but what determines which information they retain is less clear. We built a simulator in which every factor behind an image is known, so a probe can measure how readable each factor remains after training. We trained small encoders (0.7M parameters) with I-JEPA, SimCLR and BYOL on the same pairs of views and probed a factor that both views share. I-JEPA had a predictor with half the encoder's width and depth, the proportions it uses for ViT-B. Its median score was about 40% of SimCLR's. Widening the predictor to the encoder's width reversed this ordering at every training seed, against both other methods. We release the simulator and the trained models.
Related
- Inside-JEPA: From Prediction to Relational Grounding in Latent World Models
- SR-JEPA: Learning Predictive Latent State in 3D Scenes
- Pretrained Features as Teachers, Not States: Compact Latent World Models for Visual Planning
- InJePA: Intention-First Latent Planning with Coupled JEPA World Models for Visual Navigation
Source: arXiv cs.CV | 2026-10-26