EmotionDialogCN: A Spontaneous Multimodal Dataset for Mandarin Emotional Dialogue

arXiv:2608.20905v1 Announce Type: new Abstract: Face-to-face audiovisual interaction is central to human communication, conveying rich emotional and social cues. However, existing multimodal dialogue datasets remain limited by inadequate emotion annotations, poor emotional diversity, and small scale. We introduce EmotionDialogCN, a large-scale audiovisual-emotional dataset designed to capture authentic face-to-face communication. It contains 21,880 dialogue sessions performed by 119 professional actors across 20 everyday scenarios, covering 18 emotion categories with over 400 hours of recordings, the largest and most comprehensive dataset of its kind. A novel data collection framework minimizes equipment interference, enabling natural and nuanced emotional expressions. EmotionDialogCN achieves an emotion distribution deviation of 0.64 from real human emotion statistics (versus 5.65 for prior datasets) and consistent subject framing (52-59% frame occupancy). Together, these properties translate into stable unimodal and multimodal performance across acoustic, lexical, and visual modalities, with fusion results further underscoring strong multimodal alignment and cross-modal complementarity.

Related

• PolySLGen: Online Multimodal Speaking-Listening Reaction Generation in Polyadic Interaction
• EmoWorld: A Decoupled Affective Field for Controllable Emotional Video Generation
• The GENEA Challenge 2026: A Large-Scale Disentangled Evaluation of Speech-Driven Gesture Generation on the Seamless Interaction Dataset
• Beyond Accuracy: Benchmarking Cross-Task Consistency in Unified Multimodal Models

Source: arXiv cs.CV | 2026-08-24