TouchAI: Exploring human-AI perceptual alignment in touch through language model representations

arXiv:2406.06587v2 Announce Type: replace Abstract: Aligning large language models (LLMs) behaviour with human intent is critical for future AI. An important yet often overlooked aspect of this alignment is the perceptual alignment. Perceptual modalities like touch are more multifaceted and nuanced compared to other sensory modalities such as vision. This work investigates how well LLMs align with human touch experiences using the "textile hand" task. We created a "Guess What Textile" interaction in which participants were given two textile samples -- a target and a reference -- to handle. Without seeing them, participants described the differences between them to the LLM. Using these descriptions, the LLM attempted to identify the target textile by assessing similarity within its high-dimensional embedding space. Our results suggest that a degree of perceptual alignment exists, however varies significantly among different textile samples. For example, LLM predictions are well aligned for silk satin, but not for cotton denim. Moreover, participants didn't perceive their textile experiences closely matched by the LLM predictions. This is only the first exploration into perceptual alignment around touch, exemplified through textile hand. We discuss possible sources of this alignment variance, and how better human-AI perceptual alignment can benefit future everyday tasks.

Related

• Alignment Data Map for Efficient Preference Data Selection and Diagnosis
• WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback
• One Adapts to Any: Meta Reward Modeling for Personalized LLM Alignment
• Meet Dynamic Individual Preferences: Resolving Conflicting Human Value with Paired Fine-Tuning
• Machine Behavior in Relational Moral Dilemmas: Moral Rightness, Predicted Human Behavior, and Model Decisions

Source: arXiv cs.CL | 2026-04-29