Contrastive Mixed Prompt Learning for Incomplete Multimodal Sentiment Analysis with Unseen Modality Combination

arXiv:2608.20019v1 Announce Type: new Abstract: Incomplete multimodal sentiment analysis has garnered significant attention in recent years. Existing approaches typically assume that data is missing at random or are designed specifically for certain missing patterns, ignoring the modality combination inconsistency between training and testing phases. However, in real-world scenarios, the testing phase often encounters modal combinations that were not present during the training phase, which leads to insufficient generalization capabilities and unstable performance. In this paper, we introduce the problem of Incomplete Multimodal Sentiment Analysis with Unseen Modality Combinations (IMSAUMC), aiming to enhance model generalization for unseen modality combinations. To address this challenge, we propose the model named extbf{C}ontrastive extbf{M}ixed extbf{P}rompt extbf{L}earning (extsf{CMPL}) for IMSAUMC. It introduces a label-guided contrastive feature learning mechanism to learn robust and discriminative cross-modal representations. Additionally, we design modality-combination prompts with a soft router to facilitate better learning of various modality combinations. Furthermore, we introduce three prompt contrastive learning strategies, which enable effective learning of prompts corresponding to unseen modality combinations, thereby significantly strengthening the model's generalization capabilities in diverse testing scenarios. Extensive experiments on three widely used datasets demonstrate that extsf{CMPL} achieves more than a 5% improvement in accuracy compared to state-of-the-art approaches.

Related

• Residual-Guided Expert Specialization for Incomplete Multimodal Learning
• Learning What Not to Learn: Adversarial Disentangled Prompt Tuning for Robust Vision-Language Models
• QASA: Quality-Aware Semantic Augmentation for Robust Multimodal Sentiment Analysis

Source: arXiv cs.AI | 2026-08-21