Search: “multimodal”

PromptDx: Differentiable Prompt Tuning for Multimodal In-Context Alzheimer's Diagnosis

arXiv:2605.08585v1 Announce Type: cross Abstract: Deep learning models in medical imaging typically operate as parametric memory, diagnosing patients by recalling fixed knowledge learned during traini

researcharxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Reinforcing Multimodal Reasoning Against Visual Degradation

DGX agent

arXiv:2605.09262v1 Announce Type: cross Abstract: Reinforcement Learning has significantly advanced the reasoning capabilities of Multimodal Large Language Models (MLLMs), yet the resulting policies r

safetyarxiv-cs-cl

12 May 2026

RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wild

arXiv:2605.10357v1 Announce Type: cross Abstract: Multimodal misinformation increasingly leverages visual persuasion, where repurposed or manipulated images strengthen misleading text. We introduce ex

SKG-VLA: Scene Knowledge Graph Priors for Structured Scene Semantics and Multimodal Reasoning for Decision Making

arXiv:2605.09343v1 Announce Type: new Abstract: Decision making in large-scale complaint handling systems increasingly relies on heterogeneous evidence, including complaint narratives, screenshots, or

safetyarxiv-cs-ai

Agents

Through the Lens of Character: Resolving Modality-Role Interference in Multimodal Role-Playing Agent

arXiv:2605.09443v1 Announce Type: cross Abstract: The advancement of Multimodal Large Language Models (MLLMs) has expanded Role-Playing Agents (RPAs) into visually grounded environments. However, huma

agentsarxiv-cs-cl

Towards Customized Multimodal Role-Play

arXiv:2605.08129v1 Announce Type: new Abstract: Unified multimodal understanding and generation models enable richer human-AI interaction. Yet jointly customizing a character's persona, dialogue style

safetyarxiv-cs-lg

Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agents

arXiv:2605.10832v1 Announce Type: new Abstract: Multimodal deep search requires an agent to solve open-world problems by chaining search, tool use, and visual reasoning over evolving textual and visua

Efficient Data Selection for Multimodal Models via Incremental Optimization Utility

arXiv:2605.07488v1 Announce Type: new Abstract: The scaling of Large Multimodal Models (LMMs) is constrained by the quality-quantity trade-off inherent in synthetic data. Previous approaches, such as

Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models

arXiv:2602.07026v2 Announce Type: replace-cross Abstract: Despite the success of multimodal contrastive learning in aligning visual and linguistic representations, a persistent geometric anomaly, the

safetyarxiv-cs-ai

Applications

Multimodal synthesis of MRI and tabular data with diffusion in a joint latent space via cross-attention

arXiv:2605.06699v1 Announce Type: cross Abstract: We propose a multimodal latent diffusion model that jointly synthesizes volumetric magnetic resonance imaging (MRI) and tabular clinical data within a

applicationsarxiv-cs-ai

Structured Role-Aware Policy Optimization for Multimodal Reasoning

arXiv:2605.07274v1 Announce Type: new Abstract: Reinforcement learning from verifiable rewards (RLVR), especially with Group Relative Policy Optimization (GRPO), has shown strong potential for improvi

safetyarxiv-cs-ai

LUCAS-MEGA: A Large-Scale Multimodal Dataset for Representation Learning in Soil-Environment Systems

model-releasesarxiv-cs-lg

arXiv:2605.04323v1 Announce Type: new Abstract: Understanding soil is fundamental to agriculture, carbon cycling, and environmental sustainability, yet progress is limited by fragmented and heterogene

7 May 2026

To Fuse or to Drop? Dual-Path Learning for Resolving Modality Conflicts in Multimodal Emotion Recognition

arXiv:2605.04877v1 Announce Type: cross Abstract: Multimodal emotion recognition (MER) benefits from combining text, audio, and vision, yet standard fusion often fails when modalities conflict. Crucia

safetyarxiv-cs-lg

7 May 2026

MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs

arXiv:2505.20740v3 Announce Type: replace Abstract: The rapid advancement of multimodal large language models (MLLMs) offers new opportunities for complex scientific challenges, yet their application

6 May 2026

WildTableBench: Benchmarking Multimodal Foundation Models on Table Understanding In the Wild

arXiv:2605.01018v1 Announce Type: new Abstract: Using multimodal foundation models to analyze table images is a high-value yet challenging application in consumer and enterprise scenarios. Despite its

5 May 2026

Hardware

Boosting multimodal inference performance by >10% with a single Python dictionary

This Modal blog post describes a performance optimization technique for multimodal AI inference that achieves over 10% improvement through a simple Python dictionary-based approach. The article likely

hardwaremodal-blog

4 May 2026

InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?

arXiv:2604.27419v1 Announce Type: new Abstract: With the advancement of multimodal large language models (MLLMs) and coding agents, the website development has shifted from manual programming to agent

1 May 2026

JI-ADF: Joint-Individual Learning with Adaptive Decision Fusion for Multimodal Skin Lesion Classification

arXiv:2604.27343v1 Announce Type: new Abstract: Skin lesion classification is essential for early dermatological diagnosis, yet many existing computer-aided systems rely primarily on dermoscopic image

1 May 2026

VTBench: A Multimodal Framework for Time-Series Classification with Chart-Based Representations

arXiv:2604.27259v1 Announce Type: new Abstract: Time-series classification (TSC) has advanced significantly with deep learning, yet most models rely solely on raw numerical inputs, overlooking alterna

researcharxiv-cs-cv

1 May 2026

SciMDR: Advancing Scientific Multimodal Document Reasoning

arXiv:2603.12249v2 Announce Type: replace-cross Abstract: Constructing scientific multimodal document reasoning datasets for foundation model training involves an inherent trade-off among scale, faith

30 Apr 2026

Applications

Tree-of-Evidence: Efficient 'System 2' Search for Faithful Multimodal Grounding

arXiv:2604.07692v2 Announce Type: replace Abstract: Large Multimodal Models (LMMs) achieve state-of-the-art performance in high-stakes domains like healthcare, yet their reasoning remains opaque. Curr

applicationsarxiv-cs-lg

30 Apr 2026

Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence

arXiv:2604.24954v1 Announce Type: cross Abstract: We introduce Nemotron 3 Nano Omni, the latest model in the Nemotron multimodal series and the first to natively support audio inputs alongside text, i

29 Apr 2026

Toward Multimodal Conversational AI for Age-Related Macular Degeneration

arXiv:2604.25720v1 Announce Type: cross Abstract: Despite strong performance of deep learning models in retinal disease detection, most systems produce static predictions without clinical reasoning or

29 Apr 2026

VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation

arXiv:2604.25235v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly used as automated judges for multimodal systems, yet their scores provide no indication of reliability.

29 Apr 2026

Animalbooth: multimodal feature enhancement for animal subject personalization

arXiv:2509.16702v2 Announce Type: replace Abstract: Personalized animal image generation is challenging due to rich appearance cues and large morphological variability. Existing approaches often exhib

safetyarxiv-cs-cv

Can Multimodal Large Language Models Truly Understand Small Objects?

arXiv:2604.22884v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have shown promising potential in diverse understanding tasks, e.g., image and video analysis, math and physi

EmoTrans: A Benchmark for Understanding, Reasoning, and Predicting Emotion Transitions in Multimodal LLMs

arXiv:2604.23348v1 Announce Type: cross Abstract: Recent multimodal large language models (MLLMs) have shown strong capabilities in perception, reasoning, and generation, and are increasingly used in

Introducing NVIDIA Nemotron 3 Nano Omni: Long-Context Multimodal Intelligence for Documents, Audio and Video Agents

model-releaseshugging-face

NVIDIA's Nemotron 3 Nano Omni is a lightweight multimodal AI model capable of processing documents, audio, and video inputs for building intelligent agents. The model supports long-context understandi

NVIDIA Nemotron 3 Nano Omni Powers Multimodal Agent Reasoning in a Single Efficient Open Model

model-releasesnvidia-developer

NVIDIA Nemotron 3 Nano Omni is a 30B hybrid mixture-of-experts model that brings multimodal perception and reasoning into a single system, natively supporting text, image, video, and audio inputs whil

A Multimodal Text- and Graph-Based Approach for Open-Domain Event Extraction from Documents

arXiv:2604.21885v1 Announce Type: cross Abstract: Event extraction is essential for event understanding and analysis. It supports tasks such as document summarization and decision-making in emergency

researcharxiv-cs-ai

Applications

Anomaly Detection in Smart Power Grids with Graph-Regularized MS-SVDD: a Multimodal Subspace Learning Approach

arXiv:2502.15793v2 Announce Type: replace Abstract: Anomaly detection in smart power grids is a critical challenge due to the complexity, heterogeneity, and dynamic nature of sensor data streams. Exis

applicationsarxiv-cs-lg

Multimodal Protein Language Models for Enzyme Kinetic Parameters: From Substrate Recognition to Conformational Adaptation

arXiv:2603.12845v2 Announce Type: replace Abstract: Predicting enzyme kinetic parameters quantifies how efficiently an enzyme catalyzes a specific substrate under defined biochemical conditions. Canon

safetyarxiv-cs-cv

Structured Visual Narratives Undermine Safety Alignment in Multimodal Large Language Models

arXiv:2603.21697v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) extend text-only LLMs with visual reasoning, but also introduce new safety failure modes under visual

AROMA: Augmented Reasoning Over a Multimodal Architecture for Virtual Cell Genetic Perturbation Modeling

arXiv:2604.20263v1 Announce Type: cross Abstract: Virtual cell modeling predicts molecular state changes under genetic perturbations in silico, which is essential for biological mechanism studies. How

researcharxiv-cs-ai

23 Apr 2026

OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Models

arXiv:2502.16161v2 Announce Type: replace-cross Abstract: Visually-situated text parsing (VsTP) has recently seen notable advancements, driven by the growing demand for automated document understandin

22 Apr 2026

VLM Performance：Qwen3.6-27B is natively multimodal, supporting both vision-language thinking and non-thinking modes in a single unified chec…

VLM Performance：Qwen3.6-27B is natively multimodal, supporting both vision-language thinking and non-thinking modes in a single unified checkpoint — the same as Qwen3.6-35B-A3B. It handles images and

model-releasesqwen--x

22 Apr 2026

AeroRAG: Structured Multimodal Retrieval-Augmented LLM for Fine-Grained Aerial Visual Reasoning

arXiv:2604.17889v1 Announce Type: new Abstract: Despite recent progress in multimodal large language models (MLLMs), reliable visual question answering in aerial scenes remains challenging. In such sc

AutoRubric: Rubric-Based Generative Rewards for Faithful Multimodal Reasoning

arXiv:2510.14738v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have rapidly advanced from perception tasks to complex multi-step reasoning, yet reinforcement learning wit

Cognitive Chain-of-Thought (CoCoT): Structured Multimodal Reasoning about Social Situations

arXiv:2507.20409v2 Announce Type: replace Abstract: Chain-of-Thought (CoT) prompting helps models think step by step. But naive CoT breaks down in visually grounded social tasks, where models must per

Enhancing Zero-shot Personalized Image Aesthetics Assessment with Profile-aware Multimodal LLM

arXiv:2604.17233v1 Announce Type: new Abstract: Personalized image aesthetics assessment (PIAA) aims to predict an individual user's subjective rating of an image, which requires modeling user-specifi

researcharxiv-cs-cv

From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents

arXiv:2603.01455v2 Announce Type: replace-cross Abstract: While multimodal large language models have demonstrated impressive short-term reasoning, they struggle with long-horizon video understanding

MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval

model-releasesarxiv-cs-lg

arXiv:2604.18584v1 Announce Type: cross Abstract: Mathematical problem solving remains a challenging test of reasoning for large language and multimodal models, yet existing benchmarks are limited in

Mechanisms of Multimodal Synchronization: Insights from Decoder-Based Video-Text-to-Speech Synthesis

arXiv:2411.17690v3 Announce Type: replace-cross Abstract: Unified decoder-only transformers have shown promise for multimodal generation, yet the mechanisms by which they synchronize modalities with h

safetyarxiv-cs-cv

Missing-by-Design: Certifiable Modality Deletion for Revocable Multimodal Sentiment Analysis

arXiv:2602.16144v3 Announce Type: replace Abstract: As multimodal systems increasingly process sensitive personal data, the ability to selectively revoke specific data modalities has become a critical

Revisiting Change VQA in Remote Sensing with Structured and Native Multimodal Qwen Models

arXiv:2604.18429v1 Announce Type: new Abstract: Change visual question answering (Change VQA) addresses the problem of answering natural-language questions about semantic changes between bi-temporal r

Unified Multimodal Brain Decoding via Cross-Subject Soft-ROI Fusion

arXiv:2512.20249v3 Announce Type: replace-cross Abstract: Multimodal brain decoding aims to reconstruct semantic information that is consistent with visual stimuli from brain activity signals such as

When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life

arXiv:2601.04043v2 Announce Type: replace Abstract: As Multimodal Large Language Models (MLLMs) become an indispensable assistant in human life, the unsafe content generated by MLLMs poses a danger to

Local Ai

Where to Focus: Query-Modulated Multimodal Keyframe Selection for Long Video Understanding

arXiv:2604.17422v1 Announce Type: new Abstract: Long video understanding remains a formidable challenge for Multimodal Large Language Models (MLLMs) due to the prohibitive computational cost of proces

local-aiarxiv-cs-cv