Research
Continuous Conditioning of VLAs with Augmenting EMG and Visual Task Descriptors
arXiv:2610.01794v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models rely strongly on language for describing task information, despite having multimodal inputs. We hypothesize that oth
arXiv:2610.01794v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models rely strongly on language for describing task information, despite having multimodal inputs. We hypothesize that other modalities in the state space may present opportunities for supplemental task conditioning, which may be particularly relevant in cluttered or otherwise ambiguous scenes. We introduce two tuned models to test this hypothesis: (1) an electrophysiology-conditioned VLA (EC-VLA) that incorporates 8-channel electromyography envelopes as continuous conditioning input concatenated to the proprioceptive vector, and (2) a visually-annotated VLA (VA-VLA) that incorporates visual segmentation annotations to the image inputs. On a cube-selection task evaluated across three participants, EC-VLA matches a language-prompted baseline in uncluttered, in-distribution conditions and substantially outperforms it in cluttered, out-of-distribution scenes. Similarly, VA-VLA shows modest improvements over a language-prompted baseline in in-distribution scenes with substantial improvement in cluttered, out-of-distribution trials. Together, these results provide strong evidence for the potential benefit of task-conditioning beyond language.
Related
- Resolving State-Representation Mismatch: State-Space Visual Reasoning for Open-Loop VLA Planning
- Test-Time Perturbation Learning with Delayed Feedback for Vision-Language-Action Models
- StemVLA:An Open-Source Vision-Language-Action Model with Future 3D Spatial Geometry Knowledge and 4D Historical Representation
- QuoVLA: Quotient Space for Vision-Language-Action Models
Source: arXiv cs.CV | 2026-10-02