Research
Multimodal Sentiment Analysis with Missing Modality: A Knowledge-Transfer Approach
arXiv:2401.10747v5 Announce Type: replace-cross Abstract: Multimodal sentiment analysis aims to identify the emotions expressed by individuals through visual, language, and acoustic cues. However, mos
arXiv:2401.10747v5 Announce Type: replace-cross Abstract: Multimodal sentiment analysis aims to identify the emotions expressed by individuals through visual, language, and acoustic cues. However, most existing research assume that all modalities are available during both training and testing, which makes their algorithms susceptible to the missing-modality scenarios. In this paper, we propose a novel knowledge-transfer network to translate between different modalities to reconstruct the missing audio features. Moreover, we develop a cross-modality attention mechanism to maximize the information extracted from the reconstructed and observed modalities for sentiment prediction. Extensive experiments on three publicly available datasets demonstrate significant improvements over baseline methods and achieve comparable results to the previous methods with complete multi-modality supervision.
Related
- Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysis
- Collaboration of Fusion and Independence: Hypercomplex-driven Robust Multi-Modal Knowledge Graph Completion
- E2E-GMNER: End-to-End Generative Grounded Multimodal Named Entity Recognition
- MARA: A Multimodal Adaptive Retrieval-Augmented Framework for Document Question Answering
Source: arXiv cs.CL | 2026-04-21