Search: “multimodal”

Lost in Perception: Isolating Perceptual and Reasoning Failures in Multimodal Physics and Geometry Reasoning

arXiv:2609.18991v1 Announce Type: new Abstract: Multimodal LLMs report strong performance on scientific reasoning benchmarks, yet most treat perception and reasoning as a single measurable process. We

17 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Multimodal Conditioning of Fine-Tuned Stable Diffusion XL for Controllable and Culturally Faithful Ulos Motif Generation

DGX agent

arXiv:2609.17987v1 Announce Type: new Abstract: The traditional Batak Ulos weaving industry faces growing challenges in producing diverse, innovative motifs due to limitations in conventional, manuall

model-releasesarxiv-cs-ai

17 Sept 2026

Multimodal Cultural Heritage Architectural Style Classification for Residential Buildings in the UAE Based on CLIP Embeddings and SVM

arXiv:2609.17181v1 Announce Type: cross Abstract: The analysis and classification of cultural heritage architectural styles remain challenging due to the complexity of visual images of buildings, whic

16 Sept 2026

Negation Beyond the Verbal Channel: Temporal Multimodal Correlates in Dialogue

arXiv:2609.16396v1 Announce Type: new Abstract: Negation is typically modeled through its linguistic realization, although spoken interaction is accompanied by tightly coordinated nonverbal behavior.

16 Sept 2026

Unsafe by Reciprocity: How Generation-Understanding Coupling Undermines Safety in Unified Multimodal Models

arXiv:2603.27332v2 Announce Type: replace Abstract: Recent advances in Large Language Models (LLMs) and Text-to-Image (T2I) models have led to the emergence of Unified Multimodal Models (UMMs), where

safetyarxiv-cs-cv

16 Sept 2026

A Comprehensive Review of Multimodal Facial State Analysis: Tasks, Methods, and Resources

arXiv:2609.13255v1 Announce Type: new Abstract: Facial state analysis plays a crucial role in understanding human expressions, psychological modeling, and human computer interaction. Traditional unimo

A Language-Guided Multimodal Foundation Model for Zero-Shot and Multi-Task Brain Signal Analysis

arXiv:2609.15740v1 Announce Type: cross Abstract: Brain signal analysis is essential for both neuroscience research and clinical diagnostics, yet current approaches face critical limitations. End-to-e

safetyarxiv-cs-ai

ArtSociety: Multi-Agent Multimodal Collaboration for Art Emotion Understanding

arXiv:2609.13240v1 Announce Type: new Abstract: The AffectiveArt Multidimensional Art Emotion Understanding task asks to jointly predict an artwork's fine-grained emotion (12 classes, 1549:1 head-to-t

safetyarxiv-cs-cv

AURA: Unified Multimodal Framework for Conversational Music Editing

arXiv:2609.14344v1 Announce Type: cross Abstract: Instruction-guided music editors typically process each request independently, limiting their ability to support workflows in which users progressivel

Exploring Multimodal Turn-Taking Cues in Face-to-Face Conversation using Voice Activity Projection

arXiv:2609.14666v1 Announce Type: cross Abstract: Turn-taking is a fundamental component of spoken interaction, and while humans naturally rely on both verbal and non-verbal signals, dialogue systems

Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations

arXiv:2410.15359v2 Announce Type: replace Abstract: Multimodal Foundation Models represent a significant leap in artificial intelligence. Among them, Large Vision-Language Models (LVLMs) serve as the

Modality-Guided Mixture of Structured Experts with Entropy-Triggered Routing for Multimodal Recommendation

arXiv:2602.20723v3 Announce Type: replace Abstract: Multimodal recommenders combine collaborative behavior with visual and textual item evidence, whose usefulness varies across user-item interactions.

Calibrated Ambiguity in Multimodal Language Models: Humans reach for cultural references, while models describe the picture

arXiv:2609.12575v1 Announce Type: new Abstract: Ambiguity is often treated as a bug for AI systems to resolve---but in human communication and culture, ambiguity can also be a generative resource. Fro

14 Sept 2026

FINESSE: An Agent-Based Simulator and Benchmark Dataset for Multimodal Financial Event Sequences

model-releasesarxiv-cs-lg

arXiv:2609.11993v1 Announce Type: new Abstract: Machine learning research in financial services is limited by the scarcity of representative open-source datasets. Existing resources are often narrowly

14 Sept 2026

QuPAINT: Physics-Aware Multimodal Reasoning for Quantum Material Characterization

arXiv:2609.12202v1 Announce Type: new Abstract: Characterizing two-dimensional (2D) quantum materials by optical microscopy requires localizing exfoliated flakes and determining their layer thickness

14 Sept 2026

MultiHuSE: A Multimodal Dataset for Humour Styles and Emotions

arXiv:2609.11322v1 Announce Type: new Abstract: Computational recognition of verbal humour remains a challenging task, requiring an understanding of language, delivery style, emotions, and cultural co

RiVaT-Fuse: Reliability-Calibrated Variational Tensor Fusion for Multimodal Prediction under Modality Uncertainty

arXiv:2609.10798v1 Announce Type: cross Abstract: Image-metadata prediction requires fusing heterogeneous evidence whose reliability can vary across samples and latent factors. Existing representation

Robust Multimodal Sentiment Analysis with Incomplete Modalities via Semantic-aware Completeness based Reconstruction

model-releasesarxiv-cs-cl

arXiv:2609.10950v1 Announce Type: new Abstract: Recent multimodal sentiment analysis studies increasingly adopt text-centric fusion approaches to exploit the rich sentiment information inherent in the

Toward Interpretable Multimodal Fusion: Heat Conduction Modeling for Hyperspectral and LiDAR Joint Classification

arXiv:2609.11040v1 Announce Type: new Abstract: The fusion of hyperspectral (HS) and Light Detection and Ranging (LiDAR) data plays a crucial role in enhancing land-cover classification by jointly exp

A Confidence-Aware Multimodal Fusion Framework for Industrial Human-Robot Collaboration

arXiv:2609.10339v1 Announce Type: new Abstract: A confidence-aware multimodal fusion framework (CAMF) is proposed to realize reliable human intention prediction for industrial human-robot collaboratio

safetyarxiv-cs-ro

10 Sept 2026

Eliciting Self-Verification in Multimodal Reasoning Agents with Reinforcement Learning

arXiv:2609.08025v1 Announce Type: cross Abstract: Reasoning agents increasingly rely on external tools such as web search to answer complex queries. Reinforcement learning (RL) finetuning algorithms s

EntropyPrune: Matrix Entropy Guided Visual Token Pruning for Multimodal Large Language Models

arXiv:2602.17196v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) incur substantial inference cost due to the processing of hundreds of visual tokens per image. Although tok

MoEMB: Scaling Universal Multimodal Embeddings with Efficient Mixture-of-Experts Models

arXiv:2609.08663v1 Announce Type: new Abstract: Universal multimodal embedding (UME) increasingly demands encoder's capacity for handling a broad range of tasks and modalities with increased complexit

researcharxiv-cs-lg

Tutorials

TeMo: Temperature Modulation for Multimodal Contrastive Learning

arXiv:2609.07540v1 Announce Type: new Abstract: Contrastive learning approaches achieve strong performance by training models to bring similar samples closer while pushing dissimilar samples apart. A

tutorialsarxiv-cs-cv

Cross-modal triage network: a multimodal deep learning framework for severity-based triage and visual explainability in chest radiographs

arXiv:2609.04357v1 Announce Type: cross Abstract: Purpose: Increased number of chest radiograph (CXR) scans create a triage bottleneck, queueing urgent examinations behind routine ones. Existing AI to

7 Sept 2026

MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression

arXiv:2609.04947v1 Announce Type: cross Abstract: Recently, multimodal large-scale reasoning models have demonstrated remarkable capabilities in solving complex tasks through long Chains-of-Thought (M

safetyarxiv-cs-ai

7 Sept 2026

Establishing a Dynamic Multimodal HRI Dataset for Engagement Analysis with a Humanoid Robot

arXiv:2609.03255v1 Announce Type: new Abstract: This paper presents an experimental design for constructing a multimodal dataset to analyze user engagement in human-robot interaction (HRI). Prior stud

researcharxiv-cs-ro

4 Sept 2026

Occlusion-Robust Multimodal Emotion Recognition in VR via Fusion of Facial Images and EMG

arXiv:2609.03569v1 Announce Type: new Abstract: Head-mounted displays (HMDs) fundamentally limit emotion recognition in virtual reality (VR): by occluding the upper face, they render conventional imag

4 Sept 2026

VisCAD: A Foundation Model Suite with Multimodal Industrial CAD Intelligence

arXiv:2609.03811v1 Announce Type: cross Abstract: AI-assisted computer-aided design (CAD) for industrial products involves two challenging phases. Part-level generation maps diverse forms of user inte

4 Sept 2026

Deeply Interleaved Text-Image Contexts for Multimodal LLMs Assessment

arXiv:2609.02573v1 Announce Type: new Abstract: Current evaluations and training of multimodal models predominantly focus on multi-image tasks, largely overlooking interleaved text-image scenarios. In

3 Sept 2026

Predictors of Loneliness in Older Adults Using Multimodal Analysis of Speech and Language

arXiv:2609.02606v1 Announce Type: new Abstract: Loneliness is a critical public health issue among older adults, linked to higher risks of depression, cognitive decline, and mortality. Scalable, objec

3 Sept 2026

TC-Next: Zero-Shot Multimodal Cyclone Forecasting

arXiv:2609.02085v1 Announce Type: cross Abstract: We present TropicalCycloneNext (TC-Next), a multimodal deep learning model that forecasts tropical cyclone track and intensity at 6-24 h leads by leve

3 Sept 2026

Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges

arXiv:2607.19011v2 Announce Type: replace-cross Abstract: Multimodal humor in memes, cartoons, and comics remains difficult for AI systems because intended meaning depends on non-literal mechanisms, s

Evaluating Multimodal LLMs as Generalist Vision-Language-Action Agents for Drone Control: Commanding, Approaching, Tracking and Searching

arXiv:2609.01404v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are strong perceivers of images and video. We ask how far that reach extends into acting: dropping an MLLM di

Local Ai

S^2Prune: Spatially Structured Visual Token Pruning for Multimodal Large Language Models

arXiv:2609.01224v1 Announce Type: new Abstract: Visual token pruning reduces the inference overhead of multimodal large language models (MLLMs) by retaining only a subset of visual tokens. Existing me

local-aiarxiv-cs-cv

SlideMix: Enhancing Whole Slide Image Analysis via Multimodal Shuffling

arXiv:2609.00396v1 Announce Type: new Abstract: Histopathological whole slide images (WSIs) are central to cancer diagnosis, but their gigapixel scale, tissue heterogeneity, weak slide-level supervisi

3D-MRL: Nested Multimodal 3D Representations via Matryoshka Representation Learning

arXiv:2608.29285v1 Announce Type: new Abstract: Vision-Language Models align point clouds with image and text embeddings, enabling zero-shot recognition, retrieval, and open-vocabulary understanding o

Agents

CM2: Multimodal Cultural Reasoning via an Integrated Multi-Agent Framework

arXiv:2608.30498v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have shown remarkable success in STEM domains, where progress is often driven by vertical, step-by-step deducti

agentsarxiv-cs-ai

Applications

Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation

arXiv:2608.30163v1 Announce Type: cross Abstract: Real-world knowledge resides in multimodal documents, necessitating retrieval-augmented generation (RAG) for accurate question answering. However, exi

applicationsarxiv-cs-cv

HalluPrism: When Multimodal Uncertainty Should Diagnose, Not Decide

arXiv:2608.29193v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) can assign similar confidence to answers that fail for different reasons. We propose HalluPrism, a behavioral d

researcharxiv-cs-lg

Learning Where Outcomes Change:Credit-Addressable Reasoning for Multimodal Geometry

arXiv:2608.30457v1 Announce Type: cross Abstract: Multimodal geometry reasoning requires VLMs to extract precise visual relations and preserve them through multi-step deduction. Existing free-form tra

MMLongEmbed: Benchmarking Multimodal Embedding Models in Long-Context Scenarios

arXiv:2606.14747v2 Announce Type: replace-cross Abstract: Recent advancements have significantly expanded the theoretical context windows of Multimodal Embedding Models (MEMs). However, larger context

Read the Room, Read the Image: Understanding Indirect Speech Acts in Multimodal Visual Contexts

model-releasesarxiv-cs-cl

arXiv:2608.30270v1 Announce Type: new Abstract: Indirect speech acts (ISAs) require pragmatic reasoning over context, as directive intent can- not be inferred from surface form alone. Prior text-based

Toward Cultural Alignment: Human-Centered Evaluation of Multimodal AI Stories Across Five African Communities

arXiv:2608.29209v1 Announce Type: new Abstract: In this paper, we examine how well AI-generated multimodal stories align with the lived practices, relationships, language, values, and visual expectati

safetyarxiv-cs-cl

When History Is Multimodal: Rethinking Context Management for Long-Horizon Agents

arXiv:2608.29897v1 Announce Type: new Abstract: Long-horizon agents need a context manager to compress growing interaction histories into a bounded working context, via passive strategies or active st

safetyarxiv-cs-cl

Local Ai

GAAT: Geometry-Aware Alignment Transformer for Multimodal UAV Perception

arXiv:2608.27971v1 Announce Type: new Abstract: Unmanned aerial vehicle (UAV) multimodal perception integrates visible (RGB), infrared (IR), synthetic aperture radar (SAR), and depth sensors for scene

local-aiarxiv-cs-cv

31 Aug 2026

Semantic Head Specialization Guides Hybrid ViT Attention for Multimodal LLMs

model-releasesarxiv-cs-cl

arXiv:2608.28383v1 Announce Type: cross Abstract: Hybrid attention dominates frontier LLMs, yet Vision Transformers (ViTs) in multimodal LLMs lack a satisfactory hybrid design, with no consensus on wh

31 Aug 2026

Agents

GraphMemix: Query-Aware Evidence Forests for Long-Term Multimodal Agent Memory