Safety
Static In, Dynamic Out: Counterfactual Action Augmentation for Moving Object Manipulation
arXiv:2607.27890v1 Announce Type: new Abstract: Visuomotor policies have advanced on manipulation tasks where the target object stays static during execution, but real deployments break this assumptio
arXiv:2607.27890v1 Announce Type: new Abstract: Visuomotor policies have advanced on manipulation tasks where the target object stays static during execution, but real deployments break this assumption: parts drift on conveyors and fruits sway in the wind. We introduce Static In, Dynamic Out (SIDO), a counterfactual action augmentation that enables a policy trained only on static object demonstrations to adapt to unseen object motion at test time. Our key idea is to factorize moving object manipulation into two sub-problems: predicting where the object will be, and reaching that predicted pose. SIDO displaces the object to a counterfactual future position and morphs the demonstrated action chunk to preserve the hand-object relative pose, yielding a goal-conditioned policy. At deployment an object pose predictor supplies the future position. Across three simulated tasks (Mug, Square, Stack) under five object motion patterns and two real-world tasks (Gantry, Peachtree), SIDO improves moving object success over the baselines while preserving static object performance. Project website: https://sido-staticindynamicout.github.io/.
Related
- It's Not Just More Demos: Counterfactual Action Sensitivity Coverage for Data-Efficient Robust Robot Imitation
- Dynamic Execution Horizon Prediction for Chunk-based Robot Policies
- Imagine2Act: Leveraging Object-Action Motion Consistency from Imagined Goals for Robotic Manipulation
Source: arXiv cs.RO | 2026-07-31