Search: “multimodal”

An Empirical Analysis of Calibration and Selective Prediction in Multimodal Clinical Condition Classification

arXiv:2603.02719v2 Announce Type: replace Abstract: As artificial intelligence systems move toward clinical deployment, ensuring reliable prediction behavior is fundamental for safety-critical decisio

safetyarxiv-cs-lg

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Do multimodal models imagine electric sheep?

DGX agent

arXiv:2605.09693v1 Announce Type: cross Abstract: Yes. We find that large multimodal models develop mental imagery when solving spatial puzzles, and they do imagine sheep when solving sheep puzzles. W

researcharxiv-cs-ai

12 May 2026

Multimodal Emotion Recognition via Causal-Diffusion Bridge (Affect-Diff)

arXiv:2605.08252v1 Announce Type: new Abstract: Multimodal emotion recognition on CMU-MOSEI faces an extreme imbalance as Happy accounts for 65.9% of samples while three Ekman categories collectively

Personal Visual Context Learning in Large Multimodal Models

arXiv:2605.10936v1 Announce Type: new Abstract: As wearable devices like smart glasses integrate Large Multimodal Models (LMMs) into the continuous first-person visual streams of individual users, the

SalesSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators

arXiv:2605.08334v1 Announce Type: new Abstract: We present SalesSim, a framework and testbed for evaluating the ability of Multimodal Large Language Models (MLLMs) to simulate realistic, persona-drive

safetyarxiv-cs-cl

SynerMedGen: Synergizing Medical Multimodal Understanding with Generation via Task Alignment

arXiv:2605.08724v1 Announce Type: new Abstract: Unifying multimodal understanding and generation is a compelling frontier that is beginning to emerge in the medical field. However, the limited existin

safetyarxiv-cs-cv

GazeVLM: Active Vision via Internal Attention Control for Multimodal Reasoning

arXiv:2605.07817v1 Announce Type: cross Abstract: Human visual reasoning is governed by active vision, a process where metacognitive control drives top-down goal-directed attention, dynamically routin

11 May 2026

McNdroid: A Longitudinal Multimodal Benchmark for Robust Drift Detection in Android Malware

model-releasesarxiv-cs-lg

arXiv:2605.06894v1 Announce Type: cross Abstract: Machine learning (ML) in real-world systems must contend with concept drift, adversarial actors, and a spectrum of potential features with varying cos

11 May 2026

Multimodal Stepwise Clinically-Guided Attention Learning for Pathological Complete Response Prediction in Breast Cancer

arXiv:2605.07561v1 Announce Type: new Abstract: Pathological complete response (pCR) is a key prognostic factor in breast cancer patients undergoing neoadjuvant therapy, strongly associated with long-

11 May 2026

Are Multimodal LLMs Ready for Clinical Dermatology? A Real-World Evaluation in Dermatology

arXiv:2605.04098v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have demonstrated promise on publicly available dermatology benchmarks. However, benchmark performance may not

Agents

Autonomous Laparoscope Control through Unified Mechanics-Based Representation of Multimodal Intraoperative Information

arXiv:2605.04408v1 Announce Type: new Abstract: Laparoscope-holding robots can provide surgeons with a stable laparoscopic field of view (FOV) and reduce the burden on human assistants. To maintain an

agentsarxiv-cs-ro

Do Multimodal RAG Systems Leak Data? A Comprehensive Evaluation of Membership Inference and Image Caption Retrieval Attacks

arXiv:2601.17644v3 Announce Type: replace-cross Abstract: The growing adoption of multimodal Retrieval-Augmented Generation (mRAG) pipelines for vision-centric tasks (e.g., visual QA) introduces impor

applicationsarxiv-cs-ai

Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models

arXiv:2605.04874v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) has proven to be an effective solution for mitigating hallucination in Multimodal Large Language Models (MLLMs) b

safetyarxiv-cs-cl

Can Multimodal Large Language Models Understand Pathologic Movements? A Pilot Study on Seizure Semiology

arXiv:2605.03352v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated robust capabilities in recognizing everyday human activities, yet their potential for analyzi

CC-OCR V2: Benchmarking Large Multimodal Models for Literacy in Real-world Document Processing

arXiv:2605.03903v1 Announce Type: new Abstract: Large Multimodal Models (LMMs) have recently shown strong performance on Optical Character Recognition (OCR) tasks, demonstrating their promising capabi

From Static Analysis to Audience Dissemination: A Training-Free Multimodal Controversy Detection Multi-Agent Framework

model-releasesarxiv-cs-lg

arXiv:2605.02939v1 Announce Type: new Abstract: Multimodal controversy detection (MCD) identifies controversial content in videos and their associated user comments, to support risk management for soc

Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration

arXiv:2605.03820v1 Announce Type: new Abstract: Multimodal learning often grapples with the challenge of low-quality data, which predominantly manifests as two facets: modality imbalance and noisy cor

Sentinel2Cap: A Human-Annotated Benchmark Dataset for Multimodal Remote Sensing Image Captioning

arXiv:2605.03189v1 Announce Type: new Abstract: Image captioning has become an important task in computer vision, enabling models to generate natural language descriptions of visual content. While sev

Unified Multimodal Visual Tracking with Dual Mixture-of-Experts

arXiv:2605.03716v1 Announce Type: new Abstract: Multimodal visual object tracking can be divided into to several kinds of tasks (e.g. RGB and RGB+X tracking), based on the input modality. Existing met

A Multimodal Dataset for Visually Grounded Ambiguity in Machine Translation

arXiv:2605.02035v1 Announce Type: new Abstract: Ambiguity resolution is a key challenge in multimodal machine translation (MMT), where models must genuinely leverage visual input to map an ambiguous e

researcharxiv-cs-cl

Ablation Study of Multimodal Perception, Language Grounding, and Control for Human-Robot Interaction in an Object Detection and Grasping Task

arXiv:2605.00963v1 Announce Type: new Abstract: This manuscript extends our previous multimodal human-robot interaction system by introducing a controlled ablation study of the three modules that most

researcharxiv-cs-ro

BadmintonGRF: A Multimodal Dataset and Benchmark for Markerless Ground Reaction Force Estimation in Badminton

arXiv:2605.01876v1 Announce Type: new Abstract: Multimodal resources for non-periodic court sports with laboratory-grade sensing remain scarce: few publicly pair instrumented ground reaction force (GR

Disentangling Fact from Sentiment: A Dynamic Conflict-Consensus Framework for Multimodal Fake News Detection

arXiv:2512.20670v2 Announce Type: replace Abstract: Prevalent multimodal fake news detection relies on consistency-based fusion, yet this paradigm fundamentally misinterprets critical cross-modal disc

applicationsarxiv-cs-lg

GaMMA: Towards Joint Global-Temporal Music Understanding in Large Multimodal Models

arXiv:2605.00371v1 Announce Type: cross Abstract: In this paper, we propose GaMMA, a state-of-the-art (SoTA) large multimodal model (LMM) designed to achieve comprehensive musical content understandin

InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer Interaction

arXiv:2505.10887v3 Announce Type: replace Abstract: This paper introduces extsc{InfantAgent-Next}, a generalist agent capable of interacting with computers in a multimodal manner, encompassing text, i

Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE

arXiv:2605.02641v1 Announce Type: new Abstract: We present Mamoda2.5, a unified AR-Diffusion framework that seamlessly integrates multimodal understanding and generation within a single architecture.

Model-Dowser: Data-Free Importance Probing to Mitigate Catastrophic Forgetting in Multimodal Large Language Models

arXiv:2602.04509v4 Announce Type: replace Abstract: Fine-tuning Multimodal Large Language Models (MLLMs) on task-specific data is an effective way to improve performance on downstream applications. Ho

Multimodal Data Curation Through Ranked Retrieval

arXiv:2605.01163v1 Announce Type: cross Abstract: Shared embedding spaces are widely used for multimodal search and data curation. In practice, two problems often limit how well this works. First, emb

safetyarxiv-cs-lg

MultiSense-Pneumo: A Multimodal Learning Framework for Pneumonia Screening in Resource-Constrained Settings

arXiv:2605.02207v1 Announce Type: new Abstract: Pneumonia remains a leading global cause of morbidity and mortality, particularly in low resource settings where access to imaging, laboratory testing,

applicationsarxiv-cs-cv

OceanPile: A Large-Scale Multimodal Ocean Corpus for Foundation Models

arXiv:2605.00877v1 Announce Type: cross Abstract: The vast and underexplored ocean plays a critical role in regulating global climate and supporting marine biodiversity, yet artificial intelligence ha

OralMLLM-Bench: Evaluating Cognitive Capabilities of Multimodal Large Language Models in Dental Practice

arXiv:2605.01333v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have emerged as a promising paradigm for dental image analysis. However, their ability to capture the multi-lev

Referring Multiple Regions with Large Multimodal Models via Contextual Latent Steering

arXiv:2605.01827v1 Announce Type: new Abstract: Large Multimodal Models (LMMs) have recently demonstrated their proficiency in holistic visual comprehension. However, most of them struggle to tackle r

Task-Related Token Compression in Multimodal Large Language Models from an Explainability Perspective

arXiv:2506.01097v2 Announce Type: replace Abstract: Existing Multimodal Large Language Models (MLLMs) process a large number of visual tokens, leading to significant computational costs and inefficien

safetyarxiv-cs-cv

The Topology of Multimodal Fusion: Why Current Architectures Fail at Creative Cognition

arXiv:2604.04465v2 Announce Type: replace Abstract: This paper identifies a structural limitation in current multimodal AI architectures that is topological rather than parametric. Contrastive alignme

How Well Does GPT-4o Understand Vision? Evaluating Multimodal Foundation Models on Standard Computer Vision Tasks

arXiv:2507.01955v3 Announce Type: replace Abstract: Multimodal foundation models (MFMs), such as GPT-4o, have recently made remarkable progress. However, their detailed visual understanding beyond que

4 May 2026

Local Ai

Echo-{alpha}: Large Agentic Multimodal Reasoning Model for Ultrasound Interpretation

arXiv:2604.28011v1 Announce Type: new Abstract: Ultrasound interpretation requires both precise lesion localization and holistic clinical reasoning, yet existing methods typically excel at only one of

local-aiarxiv-cs-cv

EDU-CIRCUIT-HW: Evaluating Multimodal Large Language Models on Real-World University-Level STEM Student Handwritten Solutions

arXiv:2602.00095v3 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) hold significant promise for revolutionizing traditional education and reducing teachers' workload. H

From Mirage to Grounding: Towards Reliable Multimodal Circuit-to-Verilog Code Generation

arXiv:2604.27969v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) are increasingly used to translate visual artifacts into code, from UI mockups into HTML to scientific plots

K2MUSE: A human lower-limb multimodal walking dataset spanning task and acquisition variability for rehabilitation robotics

arXiv:2504.14602v2 Announce Type: replace-cross Abstract: The natural interaction and control performance of lower limb rehabilitation robots are closely linked to biomechanical information from vario

Toward Personalized Digital Twins for Cognitive Decline Assessment: A Multimodal, Uncertainty-Aware Framework

arXiv:2604.27217v1 Announce Type: new Abstract: Cognitive decline is highly heterogeneous across individuals, which complicates prognosis, trial design, and treatment planning. We present the Personal

researcharxiv-cs-ai

A Multimodal and Explainable Machine Learning Approach to Diagnosing Multi-Class Ejection Fraction from Electrocardiograms

arXiv:2604.25942v1 Announce Type: new Abstract: Left ventricular ejection fraction (LVEF) assessment depends on echocardiography, limiting access in primary care and resource-constrained settings. We

applicationsarxiv-cs-lg

30 Apr 2026

EAGLE: Expert-Augmented Attention Guidance for Tuning-Free Industrial Anomaly Detection in Multimodal Large Language Models

arXiv:2602.17419v3 Announce Type: replace Abstract: Multimodal large language models (MLLMs) can enrich industrial anomaly detection with semantic descriptions and anomaly reasoning, but they still la

EMCompress: Video-LLMs with Endomorphic Multimodal Compression

arXiv:2508.21094v3 Announce Type: replace Abstract: Video-LLMs face a fundamental tension in long-video reasoning: static, sparse frame sampling either dilutes evidence across task-irrelevant segments

FAIR_XAI: Improving Multimodal Foundation Model Fairness via Explainability for Wellbeing Assessment

arXiv:2604.23786v1 Announce Type: new Abstract: In recent years, the integration of multimodal machine learning in wellbeing assessment has offered transformative potential for monitoring mental healt

K-MetBench: A Multi-Dimensional Benchmark for Fine-Grained Evaluation of Expert Reasoning, Locality, and Multimodality in Meteorology

arXiv:2604.24645v1 Announce Type: cross Abstract: The development of practical (multimodal) large language model assistants for Korean weather forecasters is hindered by the absence of a multidimensio

MuSc-V2: Zero-Shot Multimodal Industrial Anomaly Classification and Segmentation with Mutual Scoring of Unlabeled Samples

arXiv:2511.10047v2 Announce Type: replace Abstract: Zero-shot anomaly classification (AC) and segmentation (AS) methods aim to identify and outline defects without using any labeled samples. In this p

OmniSch: A Multimodal PCB Schematic Benchmark For Structured Diagram Visual Reasoning

arXiv:2604.00270v2 Announce Type: replace Abstract: Recent large multimodal models (LMMs) have made rapid progress in visual grounding, document understanding, and diagram reasoning tasks. However, th

ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstruction

arXiv:2604.23813v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable performance in Visually Rich Document Understanding (VRDU) tasks, but their capabili