Decision-Metric Alignment in Latent World Models: Diagnostics and Action-Conditioned Objectives for MPC Planning

arXiv:2608.18746v1 Announce Type: cross Abstract: JEPA-style latent world models can use Euclidean distance to a goal latent as the cost for model-predictive control (MPC). Strong decoding of task variables, however, does not guarantee that this particular cost ranks candidate action sequences by real task progress. We call the latter property decision-metric alignment. We introduce Plan-Real Spearman, which measures latent--real rank agreement on random plans, and CEM-stage Spearman, which measures the same agreement as cross-entropy-method (CEM) search concentrates its proposal. We analyze sufficient conditions under which latent distance preserves real-cost rankings, identifying encoder distortion, terminal rollout error, and candidate margins as the controlling quantities. Guided by the observed empirical alignment gap, DA-LeWM augments LeWM with inverse-dynamics and demonstration-conditioned goal-action heads. Across all our experiments, DA-LeWM accelerates convergence and achieves higher online success than LeWM, while probe scores remain similar. These results show that action-conditioned objectives improve the geometry used by Euclidean-cost, CEM-based latent MPC.

Related

• Is Forward Prediction Enough? Physical State Grounding for JEPA World Models
• Slot-MPC: Goal-Conditioned Model Predictive Control with Object-Centric Representations
• SCALE: State-Calibrated Latent Embeddings for JEPA Planning in the Right Geometry
• SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space

Source: arXiv cs.CV | 2026-08-20