S2M-Trek: From Single to Multi-Sphere Transport via Per-Frame Deep Sets on a Wheel-Legged Robot

arXiv:2606.01332v1 Announce Type: new Abstract: We study the problem of scaling dynamic loco-manipulation from a single free-rolling sphere to multiple spheres transported simultaneously on the back of a wheel-legged quadruped, without fences, grippers, or mechanical stops. Multiple identical free-rolling spheres form an unordered set with no persistent identity: their ordering may change independently at each history frame, creating a per-frame permutation symmetry that standard history-concatenation set encoders do not explicitly enforce -- these encoders impose only a shared, diagonal permutation symmetry over the full history. We show that this symmetry mismatch leads to a concrete failure mode in curriculum-based reinforcement learning. Within the same PPO training budget, flat MLPs and branch-wise encoders plateau at or below the two-sphere stage, while a history-concatenation Deep Sets baseline (CDS) fails to progress past the two-sphere stage in our runs unless ball-to-slot assignments are randomised during training, suggesting that it exploits slot indices as a curriculum shortcut rather than learning identity-free multi-sphere dynamics. We propose extbf{Per-Frame Deep Sets (PFDS)}, which performs permutation-invariant pooling within each history frame before temporal readout; we prove that PFDS is Gframe-invariant and universally approximates continuous Gframe-invariant policies. A 2{imes}2 ablation over encoder architecture and slot randomisation separates the architectural and data-augmentation pathways, and PFDS reaches the five-sphere stage with 100% no-drop transport in simulation across all five random seeds. We further distill the PFDS teacher into TactSet via DAgger, replacing privileged sphere-state observations with a 16{imes}16 Boolean union contact map, yielding a compact and naturally Gframe-invariant tactile representation.

Source: arXiv cs.RO | 2026-06-02