Search: “multimodal”

MIS-Bench: Benchmarking Multimodal LLMs for Psychotherapeutic Interpersonal Skills Assessment

arXiv:2609.22778v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly used as evaluators, yet their reliability in professional assessment tasks that require expert

22 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

AgriScope: Pixel-Grounded Multimodal Understanding for Agricultural Images

DGX agent

arXiv:2609.20325v1 Announce Type: new Abstract: Agricultural image understanding requires fine-grained recognition of plant diseases, pests, crop structures, and botanical species under complex real-w

model-releasesarxiv-cs-cv

18 Sept 2026

Applications

From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning

arXiv:2609.19445v1 Announce Type: cross Abstract: The rapid expansion of multimodal models has surfaced formidable bottlenecks in computation, memory, and deployment, catalyzing the rise of Efficient

applicationsarxiv-cs-ai

18 Sept 2026

NeMo Data Designer: An Extensible Framework for Multimodal Synthetic Data Generation

arXiv:2609.17699v1 Announce Type: new Abstract: We present NeMo Data Designer (NDD), an open-source, general-purpose framework for multi-modal synthetic data generation (SDG). Designed to be intuitive

17 Sept 2026

Fairness at Every Intersection: Uncovering and Mitigating Intersectional Biases in Multimodal Clinical Predictions

arXiv:2412.00606v2 Announce Type: replace Abstract: Biases in automated clinical decision-making using Electronic Healthcare Records (EHR) impose significant disparities in patient care and treatment

OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation

arXiv:2609.16459v1 Announce Type: cross Abstract: Privileged on-policy distillation improves multimodal reasoning by allowing a teacher to evaluate student trajectories using rich, training-only visua

RegRet: Enhancing Region-Level Retrieval in Large Multimodal Models

arXiv:2609.16847v1 Announce Type: cross Abstract: Region-level retrieval aims to align user-specified image regions with relevant regions or textual descriptions, playing a crucial role in realworld a

Semantic-Spatial Agreement Verification for Mitigating Object Hallucination in Multimodal Large Language Models

arXiv:2609.17269v1 Announce Type: new Abstract: Multimodal large language models generate natural-language responses from visual inputs, yet may mention objects absent from an image. In medication ass

safetyarxiv-cs-cv

Towards Scalable RLVR: Multimodal Instruction Following Data Synthesis and Distillation

arXiv:2609.16059v1 Announce Type: new Abstract: Multimodal instruction following (MMIF) is crucial for building generalist agents. However, current training paradigms rely heavily on Supervised Fine-T

safetyarxiv-cs-cl

Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration

arXiv:2609.15669v1 Announce Type: cross Abstract: Multimodal image registration is a key component of many clinical workflows, yet it remains challenging because corresponding anatomical structures of

Beyond Numerical Time Series: A Unified Benchmark for Multimodal Forecasting with Heterogeneous Context

arXiv:2609.15087v1 Announce Type: cross Abstract: Most time series forecasting benchmarks remain numerical-centric and provide limited support for evaluating contextual information that shapes real-wo

Func-R1: Incentivizing Mathematical Function Reasoning in Multimodal Large Language Models

arXiv:2609.14779v1 Announce Type: new Abstract: Performing deliberate mathematical reasoning in visual contexts is a hallmark of advanced Multimodal Large Language Models (MLLMs) and requires a sophis

SHIFT-M3: Pre-fusion Alignment-based Consistency Screening for Multimodal ECG Record Integrity

arXiv:2609.13874v1 Announce Type: new Abstract: Multimodal clinical AI typically assumes that the waveform, report, metadata, and downstream predictions attached to a record belong to the same patient

safetyarxiv-cs-cl

StepPrune: Adaptive Sequential Visual Token Selection across Multimodal Large Language Models

arXiv:2609.13804v1 Announce Type: new Abstract: Visual prefixes account for a major portion of the per-layer computation in multimodal large language models (MLLMs), making visual-token pruning a dire

researcharxiv-cs-cv

LatentVerse: A Framework for Understanding Shared and Modality-Specific Information in Multimodal Latent Representations

arXiv:2609.12364v1 Announce Type: new Abstract: Latent embeddings have become a central data abstraction in modern machine learning, especially in biomedicine, where foundation models are increasingly

researcharxiv-cs-lg

14 Sept 2026

Exploring Diffusion Transformers for Cross-Modal Augmentation in Multimodal Brain State Decoding

arXiv:2609.11341v1 Announce Type: new Abstract: Multimodal brain state decoding has largely focused on fusing paired modalities for prediction, but has rarely explored how their correspondence can be

researcharxiv-cs-ai

12 Sept 2026

RAMamba-Net: A Reliability-Aware and Mamba-Based Multimodal Fusion Network for Auditory Attention Detection

arXiv:2609.11372v1 Announce Type: new Abstract: Auditory attention decoding (AAD) identifies the attended speaker from physiological signals, supporting neuro-steered hearing devices and natural human

12 Sept 2026

Leveraging LLMs for Context-Aware Implicit Textual and Multimodal Hate Speech Detection

arXiv:2510.15685v2 Announce Type: replace Abstract: This paper investigates the use of an LLM to generate auxiliary background context for social media posts, and explores four methods to incorporate

researcharxiv-cs-cl

11 Sept 2026

MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation

arXiv:2511.13135v3 Announce Type: replace Abstract: Medical vision-language models (VLMs) are increasingly expected to support clinical workflows through diagnostic text and relevant medical images. H

11 Sept 2026

EEGBind: Detecting Source-Level Interictal Epileptiform Discharges via EEG-Centric Multimodal Binding

model-releasesarxiv-cs-lg

arXiv:2609.09728v1 Announce Type: new Abstract: Source-level analysis of interictal epileptiform discharges (IEDs) is relevant to presurgical evaluation and treatment planning because it helps charact

10 Sept 2026

CNsEMD: An Expert-Annotated Multi-Field-Strength MRI Dataset and a Hyperspherical Manifold Network for Multimodal Cranial Nerve Parcellation

arXiv:2609.07058v1 Announce Type: new Abstract: Cranial nerves (CNs) play essential roles in sensory, motor, and autonomic functions. Accurate CN parcellation from multimodal magnetic resonance imagin

9 Sept 2026

RAFM-SER++: A Lightweight Multimodal Emotion Recognition Framework for Real-Time Behavioral Monitoring in Surveillance Systems

arXiv:2609.07409v1 Announce Type: cross Abstract: Recent multimodal Speech Emotion Recognition (SER) systems achieve high accuracy through interaction-heavy cross-modal transformers, but their computa

safetyarxiv-cs-lg

9 Sept 2026

Applications

Enhancing Multimodal Emotion Recognition via Multi-Feature Encoding and Attention-Based Fusion

arXiv:2609.04690v1 Announce Type: cross Abstract: Multimodal emotion recognition has attracted growing interest due to its importance in human-computer interaction, remote education, and healthcare. T

applicationsarxiv-cs-ai

7 Sept 2026

InSituMeasure: Probing Situated Measurement Grounding in Industrial Scenes with Multimodal Large Language Models

arXiv:2609.04014v1 Announce Type: new Abstract: For trained operators, gauge reading requires little specialized knowledge, low cognitive effort, and high repeatability. Yet Multimodal Large Language

4 Sept 2026

NeoRed: A Knowledge-Logic-Alignment Multimodal Large Language Model for Neonatal Respiratory Disease Diagnosis

arXiv:2609.03527v1 Announce Type: new Abstract: Neonatal respiratory diseases are a major cause of neonatal morbidity and mortality, posing substantial challenges in clinical practice. Despite recent

4 Sept 2026

Pailitao-MMSearch: Building Native E-Commerce Multimodal Search Foundation

arXiv:2607.17499v2 Announce Type: replace Abstract: The evolution of e-commerce has fundamentally transformed how users search for products, shifting from simple text-based keyword queries to complex

3 Sept 2026

Local Ai

Causal Probing for Internal Visual Representations in Multimodal Large Language Models

arXiv:2605.05593v2 Announce Type: replace Abstract: Despite the remarkable success of Multimodal Large Language Models (MLLMs) across diverse tasks, the internal mechanisms governing how they encode a

local-aiarxiv-cs-ai

2 Sept 2026

Contribution-Aware Bandwidth Allocation for Multimodal Split Learning

arXiv:2609.01406v1 Announce Type: new Abstract: Multimodal models are increasingly the default option for perception at the network edge, yet they are trained almost entirely in the datacenter, becaus

safetyarxiv-cs-lg

2 Sept 2026

Accelerating Unified Multimodal Models with Core-Expansion Routing and Unified Computation Scheduling

arXiv:2608.29291v1 Announce Type: new Abstract: Unified multimodal models jointly support understanding and generation, but incur substantial redundant computation across tokens, layers, and generatio

MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents

arXiv:2508.13186v2 Announce Type: replace-cross Abstract: AI agents with advanced reasoning and tool-use capabilities have demonstrated impressive performance in web browsing for deep search. However,

Multimodal Shared Latent Representation of Narration, Microscope and iOCT Images for Phase Recognition in Vitreoretinal Surgery

arXiv:2608.31065v1 Announce Type: new Abstract: Surgical phase recognition is key to context-aware computer-assisted feedback in vitreoretinal procedures, yet the scarcity of synchronized multimodal i

safetyarxiv-cs-cv

SnapBench: Benchmarking Snap-and-Ask Multimodal Retrieval for Mobile Interactions

arXiv:2608.29607v1 Announce Type: new Abstract: Mobile AI acts as a visual oracle, empowering users to snap a picture of something and ask for information. Snap-and-ask retrieval is now one of the mos

UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models

arXiv:2602.08336v3 Announce Type: replace Abstract: Unified multimodal models (UMMs) aim to integrate multimodal understanding and generation within a unified architecture, yet it remains unclear to w

When Models Hear What They Expect: Diagnosing Prosodic Heuristics in Multimodal Sarcasm Detection

arXiv:2608.30204v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) process speech and text jointly, yet whether they exploit prosodic cues for pragmatic inference or rely on surf

SciReC: Diagnostic Evaluation of Multimodal, Multi-Turn Relational Reasoning with Adaptive Interaction

arXiv:2608.27461v1 Announce Type: new Abstract: Relational reasoning requires the process of perceptual understanding, comparing, and integrating the underlying relationships between concepts. This ab

31 Aug 2026

Steering Multimodal Large Language Models Decoding for Context-Aware Safety

arXiv:2509.19212v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in real-world applications, yet their ability to make context-aware safety decisi

safetyarxiv-cs-cl

31 Aug 2026

Graph-Based Pseudo-multimodal Contrastive Learning for 12-Lead ECG Representations

arXiv:2608.26964v1 Announce Type: new Abstract: 12-lead electrocardiogram (ECG) is a standard, non-invasive examination widely used for diagnosing coronary artery disease, where clinical interpretatio

researcharxiv-cs-lg

28 Aug 2026

Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation

arXiv:2608.26535v1 Announce Type: new Abstract: Audio-video generation is rapidly moving from prompt-driven synthesis toward multimodal conditioning, where text, images, audio, and video can jointly s

28 Aug 2026

Said Aloud, Read Different: Cross-Modal Instability in Multimodal Models

arXiv:2608.27135v1 Announce Type: new Abstract: Multimodal foundation models are increasingly used in speech-first assistants that must interpret spoken queries and produce visually grounded decisions

28 Aug 2026

Agents

VISA: Agentic Self-Evolving Data Synthesis for Multimodal Instruction Following

arXiv:2608.26013v1 Announce Type: new Abstract: Multimodal instruction-following models require training data that is accurate, diverse, verifiable, and challenging. Existing synthesis pipelines typic

agentsarxiv-cs-cl

27 Aug 2026

CHIMERA Challenge: Biochemical Recurrence Prediction in Prostate Cancer Patients using multimodal datasets

arXiv:2608.21497v1 Announce Type: cross Abstract: Biochemical recurrence (BCR), defined as any detectable prostate-specific antigen level after prostatectomy with confirmatory elevation, is widely use

25 Aug 2026

From Recognition to Reasoning: Advancing Multimodal Harmful Meme Detection via Chain-of-Thought Alignment

arXiv:2506.18919v5 Announce Type: replace-cross Abstract: As a multimodal communication medium that integrates images and text, memes often convey implicit harmful content through metaphors, satire, a

25 Aug 2026

Applications

Multimodal Prompt Learning with Irregular EHRs for Robust Monitoring of Critical Care Patients

arXiv:2608.21941v1 Announce Type: new Abstract: Accurate assessment of patients in intensive care units (ICUs) is essential for timely clinical intervention and improved patient outcomes. Multimodal e

applicationsarxiv-cs-ai

25 Aug 2026

Agents

Vis-Poison: Poisoning Visual Knowledge in Multimodal Retrieval-Augmented Generation

arXiv:2608.20756v1 Announce Type: cross Abstract: While multimodal retrieval-augmented generation (RAG) systems increasingly rely on images as external knowledge sources, the introduction of poisoned

agentsarxiv-cs-ai

24 Aug 2026

Explainable Multimodal Depression Recognition in Clinical Interviews via PHQ-Aligned Symptom Summarization

arXiv:2501.16106v2 Announce Type: replace Abstract: Recent advances in multimodal depression recognition for clinical interviews (MDRC) have demonstrated the potential of AI systems by integrating tex

researcharxiv-cs-cl

21 Aug 2026

Holtercare-Bench: A Multimodal Benchmark for Evaluating Long-Term Dynamic ECG Analysis

model-releasesarxiv-cs-lg

arXiv:2608.19297v1 Announce Type: new Abstract: While multimodal large language models (MLLMs) excel in medical applications, most of them favor static images or short-term signals. In the critical fi

21 Aug 2026

NepOOC-M: Bilingual Nepali-English Benchmark and Comparative Analysis of Multimodal Architectures for OOC Detection

arXiv:2608.19212v1 Announce Type: new Abstract: Out-of-context (OOC) misinformation pairs authentic images with misleading captions to construct false narratives without image manipulation, making det

21 Aug 2026

MAVEN: A Macro-Societal Value Evaluation Framework of Multimodal Content with Compact Aligned Evaluators

arXiv:2608.18096v1 Announce Type: new Abstract: Assessing whether multimodal content aligns with macro-societal values, such as peace, justice, and freedom, has become an increasingly urgent challenge