Search: “multimodal”

CLIMB: Confidence-Guided Complementary Evidence for Multimodal Retrieval-Augmented Generation

arXiv:2610.03421v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have shown strong visual reasoning abilities, but knowledge-intensive visual question answering often requires

safetyarxiv-cs-cl

5 Oct 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wild

DGX agent

arXiv:2407.12927v4 Announce Type: replace Abstract: Systems for multimodal emotion recognition (ER) are commonly trained to extract features from different modalities (e.g., visual, audio, and textual

model-releasesarxiv-cs-cv

5 Oct 2026

VisAudit: Evaluating Multimodal Agents for Visual Diagnosis and Repair

model-releasesarxiv-cs-lg

arXiv:2610.02399v1 Announce Type: new Abstract: Multimodal agents are increasingly used for data visualization tasks but remain limited in autonomous review. Unlike humans, they may fail to recognize

5 Oct 2026

LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation

arXiv:2610.00333v1 Announce Type: cross Abstract: Multimodal on-policy distillation (OPD) aims to improve visual grounding while preserving the strong reasoning capabilities of language models. Recent

2 Oct 2026

Prototype-guided Bilateral Alignment Multimodal Federated Learning

arXiv:2609.38925v1 Announce Type: new Abstract: Multimodal federated learning (MFL) has emerged as a pivotal paradigm for leveraging distributed data to enhance model performance. However, existing me

1 Oct 2026

SceneJail: Exploiting Video Scenario Context to Jailbreak Multimodal LLMs

arXiv:2609.38899v1 Announce Type: cross Abstract: Video Multimodal Large Language Models (Video-MLLMs) support reasoning over video inputs, yet remain vulnerable to jailbreak attacks that elicit polic

1 Oct 2026

GeoOutageBench: Benchmarking Ambiguity-aware, Ontology-grounded Geospatiotemporal KGQA for Multimodal Power Outage and Resilience Analysis

arXiv:2609.36082v1 Announce Type: new Abstract: We introduce GeoOutageBench, a benchmark for assessing LLM-based geospatiotemporal KGQA for multimodal outage and resilience analysis. Unlike existing K

30 Sept 2026

ReCAP: Retrieval-Guided Capability Reuse for Multimodal Continual Instruction Tuning

arXiv:2609.37889v1 Announce Type: new Abstract: Multimodal continual instruction tuning (MCIT) aims to enable multimodal large language models to acquire new capabilities from sequential tasks while p

30 Sept 2026

ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Models

arXiv:2609.36562v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) are increasingly deployed in safety-critical domains, their reliability is threatened by multimodal imp

30 Sept 2026

Tutorials

How Far Are We from Removing the Visual Encoder? Scaling Laws for Encoder-Free Multimodal Pretraining

arXiv:2609.35457v1 Announce Type: new Abstract: Most modern multimodal large language models (MLLMs) build on a pretrained visual encoder that provides a strong visual prior. Encoder-free MLLMs instea

tutorialsarxiv-cs-cv

29 Sept 2026

Structured Latent Modeling for Supervised Multimodal Information Decomposition

arXiv:2609.35502v1 Announce Type: new Abstract: Multimodal prediction relies on diverse forms of evidence: information repeated across modalities, cues specific to a single source, and complex cross-m

researcharxiv-cs-lg

29 Sept 2026

ChemMLLM: Chemical Multimodal Large Language Model

model-releasesarxiv-cs-lg

arXiv:2505.16326v3 Announce Type: replace Abstract: Recent years have seen rapid progress in multimodal large language models (MLLMs) in the field of chemistry. However, chemical MLLMs that can handle

28 Sept 2026

MM-VeriAgent: Learning to Use Extensive Tools to Verify Multimodal Misinformation with Reinforcement Learning

arXiv:2609.30698v1 Announce Type: new Abstract: Real-world multimodal misinformation often involves mixed forgery sources, requiring sample-specific detection strategies. Existing tool-augmented metho

safetyarxiv-cs-cv

28 Sept 2026

Multimodal Voice Activity Projection for Social Robot Mediation: Expected Behavior and Deployment Constraints

arXiv:2609.28317v1 Announce Type: new Abstract: Turn-taking prediction is especially relevant for social robots that act as mediators in human-human interaction, where the expected action is often not

researcharxiv-cs-ro

24 Sept 2026

PhyMo: A Physical-Field Modality for Multimodal AI4Physics

arXiv:2609.27554v1 Announce Type: cross Abstract: Multimodal learning is emerging as a powerful paradigm for AI for Physics (AI4Physics), where predicting physical systems requires the joint interpret

researcharxiv-cs-ai

24 Sept 2026

VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark

arXiv:2508.13680v5 Announce Type: replace Abstract: We introduce VMMU, a Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark designed to evaluate how vision-language models (VLMs) in

24 Sept 2026

FusionMMT: A Unified Multimodal and Multitask Learning Framework for Nuclear Fusion

arXiv:2609.26095v1 Announce Type: cross Abstract: With the growing global demand for energy, nuclear fusion has emerged as a promising direction for future clean energy. Tokamaks represent one of the

23 Sept 2026

MMAP: Multimodal Missing-Aware Pretraining for Longitudinal Alzheimer's Prediction

arXiv:2609.26617v1 Announce Type: cross Abstract: Clinical decision making heavily relies on predicting the disease progression trajectory by seeking to understand patient's health status which is cha

safetyarxiv-cs-lg

23 Sept 2026

Cognitive Action Reasoning for Proactive Robots from Human-Centered Multimodal Observations

arXiv:2609.23486v1 Announce Type: cross Abstract: Robots operating in human-centered environments are typically designed to execute explicit instructions, and most robot-learning datasets likewise pai

Dissecting Training-Free Uncertainty Estimation in Multimodal Large Language Models

arXiv:2609.22206v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable performance across a wide range of multimodal tasks, yet understanding and quantifying

Enabling Vision and Cross-Modal Learning for Multimodal Stroke Recurrence Prediction: An Interpretable Two-Step Framework

arXiv:2609.22271v1 Announce Type: new Abstract: Multimodal stroke recurrence prediction requires effective integration of heterogeneous clinical and imaging data, yet modality imbalance often causes m

MIS-Bench: Benchmarking Multimodal LLMs for Psychotherapeutic Interpersonal Skills Assessment

arXiv:2609.22778v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly used as evaluators, yet their reliability in professional assessment tasks that require expert

AgriScope: Pixel-Grounded Multimodal Understanding for Agricultural Images

arXiv:2609.20325v1 Announce Type: new Abstract: Agricultural image understanding requires fine-grained recognition of plant diseases, pests, crop structures, and botanical species under complex real-w

18 Sept 2026

Applications

From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning

arXiv:2609.19445v1 Announce Type: cross Abstract: The rapid expansion of multimodal models has surfaced formidable bottlenecks in computation, memory, and deployment, catalyzing the rise of Efficient

applicationsarxiv-cs-ai

18 Sept 2026

NeMo Data Designer: An Extensible Framework for Multimodal Synthetic Data Generation

arXiv:2609.17699v1 Announce Type: new Abstract: We present NeMo Data Designer (NDD), an open-source, general-purpose framework for multi-modal synthetic data generation (SDG). Designed to be intuitive

17 Sept 2026

Fairness at Every Intersection: Uncovering and Mitigating Intersectional Biases in Multimodal Clinical Predictions

arXiv:2412.00606v2 Announce Type: replace Abstract: Biases in automated clinical decision-making using Electronic Healthcare Records (EHR) impose significant disparities in patient care and treatment

OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation

arXiv:2609.16459v1 Announce Type: cross Abstract: Privileged on-policy distillation improves multimodal reasoning by allowing a teacher to evaluate student trajectories using rich, training-only visua

RegRet: Enhancing Region-Level Retrieval in Large Multimodal Models

arXiv:2609.16847v1 Announce Type: cross Abstract: Region-level retrieval aims to align user-specified image regions with relevant regions or textual descriptions, playing a crucial role in realworld a

Semantic-Spatial Agreement Verification for Mitigating Object Hallucination in Multimodal Large Language Models

arXiv:2609.17269v1 Announce Type: new Abstract: Multimodal large language models generate natural-language responses from visual inputs, yet may mention objects absent from an image. In medication ass

safetyarxiv-cs-cv

Towards Scalable RLVR: Multimodal Instruction Following Data Synthesis and Distillation

arXiv:2609.16059v1 Announce Type: new Abstract: Multimodal instruction following (MMIF) is crucial for building generalist agents. However, current training paradigms rely heavily on Supervised Fine-T

safetyarxiv-cs-cl

Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration

arXiv:2609.15669v1 Announce Type: cross Abstract: Multimodal image registration is a key component of many clinical workflows, yet it remains challenging because corresponding anatomical structures of

Beyond Numerical Time Series: A Unified Benchmark for Multimodal Forecasting with Heterogeneous Context

arXiv:2609.15087v1 Announce Type: cross Abstract: Most time series forecasting benchmarks remain numerical-centric and provide limited support for evaluating contextual information that shapes real-wo

Func-R1: Incentivizing Mathematical Function Reasoning in Multimodal Large Language Models

arXiv:2609.14779v1 Announce Type: new Abstract: Performing deliberate mathematical reasoning in visual contexts is a hallmark of advanced Multimodal Large Language Models (MLLMs) and requires a sophis

SHIFT-M3: Pre-fusion Alignment-based Consistency Screening for Multimodal ECG Record Integrity

arXiv:2609.13874v1 Announce Type: new Abstract: Multimodal clinical AI typically assumes that the waveform, report, metadata, and downstream predictions attached to a record belong to the same patient

safetyarxiv-cs-cl

StepPrune: Adaptive Sequential Visual Token Selection across Multimodal Large Language Models

arXiv:2609.13804v1 Announce Type: new Abstract: Visual prefixes account for a major portion of the per-layer computation in multimodal large language models (MLLMs), making visual-token pruning a dire

researcharxiv-cs-cv

LatentVerse: A Framework for Understanding Shared and Modality-Specific Information in Multimodal Latent Representations

arXiv:2609.12364v1 Announce Type: new Abstract: Latent embeddings have become a central data abstraction in modern machine learning, especially in biomedicine, where foundation models are increasingly

researcharxiv-cs-lg

14 Sept 2026

Exploring Diffusion Transformers for Cross-Modal Augmentation in Multimodal Brain State Decoding

arXiv:2609.11341v1 Announce Type: new Abstract: Multimodal brain state decoding has largely focused on fusing paired modalities for prediction, but has rarely explored how their correspondence can be

researcharxiv-cs-ai

12 Sept 2026

RAMamba-Net: A Reliability-Aware and Mamba-Based Multimodal Fusion Network for Auditory Attention Detection

arXiv:2609.11372v1 Announce Type: new Abstract: Auditory attention decoding (AAD) identifies the attended speaker from physiological signals, supporting neuro-steered hearing devices and natural human

12 Sept 2026

Leveraging LLMs for Context-Aware Implicit Textual and Multimodal Hate Speech Detection

arXiv:2510.15685v2 Announce Type: replace Abstract: This paper investigates the use of an LLM to generate auxiliary background context for social media posts, and explores four methods to incorporate

researcharxiv-cs-cl

11 Sept 2026

MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation

arXiv:2511.13135v3 Announce Type: replace Abstract: Medical vision-language models (VLMs) are increasingly expected to support clinical workflows through diagnostic text and relevant medical images. H

11 Sept 2026

EEGBind: Detecting Source-Level Interictal Epileptiform Discharges via EEG-Centric Multimodal Binding

model-releasesarxiv-cs-lg

arXiv:2609.09728v1 Announce Type: new Abstract: Source-level analysis of interictal epileptiform discharges (IEDs) is relevant to presurgical evaluation and treatment planning because it helps charact

10 Sept 2026

CNsEMD: An Expert-Annotated Multi-Field-Strength MRI Dataset and a Hyperspherical Manifold Network for Multimodal Cranial Nerve Parcellation

arXiv:2609.07058v1 Announce Type: new Abstract: Cranial nerves (CNs) play essential roles in sensory, motor, and autonomic functions. Accurate CN parcellation from multimodal magnetic resonance imagin

9 Sept 2026

RAFM-SER++: A Lightweight Multimodal Emotion Recognition Framework for Real-Time Behavioral Monitoring in Surveillance Systems

arXiv:2609.07409v1 Announce Type: cross Abstract: Recent multimodal Speech Emotion Recognition (SER) systems achieve high accuracy through interaction-heavy cross-modal transformers, but their computa

safetyarxiv-cs-lg

9 Sept 2026

Applications

Enhancing Multimodal Emotion Recognition via Multi-Feature Encoding and Attention-Based Fusion

arXiv:2609.04690v1 Announce Type: cross Abstract: Multimodal emotion recognition has attracted growing interest due to its importance in human-computer interaction, remote education, and healthcare. T

applicationsarxiv-cs-ai

7 Sept 2026

InSituMeasure: Probing Situated Measurement Grounding in Industrial Scenes with Multimodal Large Language Models

arXiv:2609.04014v1 Announce Type: new Abstract: For trained operators, gauge reading requires little specialized knowledge, low cognitive effort, and high repeatability. Yet Multimodal Large Language

4 Sept 2026

NeoRed: A Knowledge-Logic-Alignment Multimodal Large Language Model for Neonatal Respiratory Disease Diagnosis

arXiv:2609.03527v1 Announce Type: new Abstract: Neonatal respiratory diseases are a major cause of neonatal morbidity and mortality, posing substantial challenges in clinical practice. Despite recent

4 Sept 2026

Pailitao-MMSearch: Building Native E-Commerce Multimodal Search Foundation

arXiv:2607.17499v2 Announce Type: replace Abstract: The evolution of e-commerce has fundamentally transformed how users search for products, shifting from simple text-based keyword queries to complex

3 Sept 2026

Local Ai

Causal Probing for Internal Visual Representations in Multimodal Large Language Models