A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Robotics

arXiv:2604.15215v1 Announce Type: new Abstract: We present a novel hierarchical spatiotemporal action tokenizer for in-context imitation learning. We first propose a hierarchical approach, which consists of two successive levels of vector quantization. In particular, the lower level assigns input actions to fine-grained subclusters, while the higher level further maps fine-grained subclusters to clusters. Our hierarchical approach outperforms the non-hierarchical counterpart, while mainly exploiting spatial information by reconstructing input actions. Furthermore, we extend our approach by utilizing both spatial and temporal cues, forming a hierarchical spatiotemporal action tokenizer, namely HiST-AT. Specifically, our hierarchical spatiotemporal approach conducts multi-level clustering, while simultaneously recovering input actions and their associated timestamps. Finally, extensive evaluations on multiple simulation and real robotic manipulation benchmarks show that our approach establishes a new state-of-the-art performance in in-context imitation learning.

Related

• ScoRe-Flow: Complete Distributional Control via Score-Based Reinforcement Learning for Flow Matching
• Model-Based Reinforcement Learning Exploits Passive Body Dynamics for High-Performance Biped Robot Locomotion
• DA-PTQ: Drift-Aware Post-Training Quantization for Efficient Vision-Language-Action Models
• Iteratively Learning Muscle Memory for Legged Robots to Master Adaptive and High Precision Locomotion
• A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Model

Source: arXiv cs.RO | 2026-04-17