EgoGenesis: Egocentric World-Action Modeling with Online Anchored Projective Memory and Action-3D RoPE

arXiv:2607.28243v1 Announce Type: new Abstract: Egocentric video offers rich manipulation experience for embodied AI, yet collecting diverse egocentric data across scenes, objects, motions, and embodiments remains costly. We present method, an egocentric world-action simulator that synthesizes controllable, high-quality manipulation videos to expand scarce real-world training data. method{} builds on a pretrained video generation prior and introduces two geometry-aware conditioning mechanisms. Online Anchored Projective Memory (OAPM) preserves a first-frame 3D scene anchor while periodically refreshing a recent state during autoregressive generation. Action-3D Rotary Position Embedding (A3D-RoPE) encodes end-effector motion with camera-aware 3D rotary coordinates, injecting action geometry into skeleton-to-video cross-attention for precise control. Together, these components improve visual fidelity, geometric stability, and action alignment in long egocentric rollouts. Moreover, augmenting 400 real trajectories with 400 method-generated trajectories improves out-of-distribution real-robot success from 77% to 84% on single-arm tasks and from 53% to 70% on dual-arm tasks, demonstrating that the synthesized data substantially improve downstream WAM generalization.

Related

• MemoryVAM: Integrating Memory into Video Action Model for Robot Manipulation
• EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World
• EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data
• Wh0: Generative World Models as Scalable Sources of Egocentric Human Hand Manipulation Data

Source: arXiv cs.CV | 2026-07-31