Uncertainty-Guided Multimodal Consensus for Multimodal Sentiment Analysis

arXiv:2510.20256v2 Announce Type: replace-cross Abstract: Multimodal sentiment analysis (MSA) has made considerable progress in recent years. Existing MSA methods mainly learn shared multimodal representations or train unimodal predictors with a common multimodal label. However, assigning the shared label to every modality can obscure modality-specific sentiment, while direct fusion may overemphasize unreliable predictions when modalities disagree. To address these issues, we propose Uncertainty-Guided Multimodal Consensus (UMC), which first recovers continuous unimodal pseudo-labels and then forms a reliability-aware consensus across modalities. Our method consists of two key components: i) Continuous Prototype-Guided Label Recovery (CPLR), which combines heteroscedastic predictions with ordered sentiment prototypes and uses confidence and label conflict to refine continuous unimodal pseudo-labels, and ii) Reliability and Precision Consensus (RPC), which estimates sample-dependent modality reliability from predictive statistics and cross-modal disagreement and combines it with predictive precision through a closed-form consensus solver. Extensive experiments conducted on four publicly available MSA datasets demonstrate the effectiveness and superiority of our proposed model.

Related

• Contrastive Mixed Prompt Learning for Incomplete Multimodal Sentiment Analysis with Unseen Modality Combination
• See, Think, Learn: A Self-Taught Multimodal Reasoner
• Learning Self-Correction in Vision-Language Models via Rollout Augmentation

Source: arXiv cs.CL | 2026-10-28