TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulation

arXiv:2603.12665v3 Announce Type: replace Abstract: Vision-Language-Action (VLA) models have demonstrated significant advantages in robotic manipulation. However, their reliance on vision and language often leads to suboptimal performance in tasks involving visual occlusion, fine-grained manipulation, and physical contact. To address these challenges, we propose TacVLA, a fine-tuned VLA model by incorporating tactile modalities into the transformer-based policy to enhance fine-grained manipulation capabilities. Specifically, we introduce a contact-aware gating mechanism that selectively activates tactile tokens only when contact is detected, enabling adaptive multimodal fusion while avoiding irrelevant tactile interference. The fused visual, language, and tactile tokens are jointly processed within the transformer architecture to strengthen cross-modal grounding during contact-rich interaction. Extensive experiments on constraint-locked disassembly, in-box picking and robustness evaluations demonstrate that TacVLA outperforms baselines, %including existing VLA models and diffusion policies, improving the performance by averaging 20% success rate in disassembly and 60% in in-box picking, achieving a 2.1imes improvement under visual occlusion, and showing recovery behavior under human disturbance. Videos are available at https://sites.google.com/view/tacvla.

Related

• Learning When to See and When to Feel: Adaptive Vision-Torque Fusion for Contact-Aware Manipulation
• TORL-VLA: Tactile Guided Online Reinforcement Learning for Contact-Rich Manipulation
• ViTacPhys: Physical Property-Aware Grasping from Human Visual-Tactile Demonstrations
• HapticVLA: Contact-Rich Manipulation via Vision-Language-Action Model without Inference-Time Tactile Sensing

Source: arXiv cs.RO | 2026-09-02