Search: “multimodal”

Omni-Perception Policy Optimization for Multimodal Emotion Reasoning

arXiv:2606.25325v1 Announce Type: new Abstract: We find that current emotion-oriented Omni-MLLMs still lack reliable omni-modal perception: they (i) underutilize multimodal cues in their reasoning tra

25 Jun 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

ReMMD: Realistic Multilingual Multi-Image Agentic Verification for Multimodal Misinformation Detection

DGX agent

arXiv:2606.24112v1 Announce Type: new Abstract: Multimodal misinformation detection is increasingly important because viral posts now combine long multilingual narratives, several images, mixed proven

model-releasesarxiv-cs-ai

24 Jun 2026

An approach with Visual and Tabular Mamba to multimodal medical data using Mixed Fusion

arXiv:2606.20738v1 Announce Type: new Abstract: This article presents a complementary approach for integrating multimodal medical data in cancer classification, based on state space models represented

Attention-Spectrum Regularization for Replay-Free Continual Multimodal LLMs

arXiv:2606.23063v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly required to adapt to non-stationary streams of visual domains, question types, and user instru

Physiology-Aware CNN and Zero-Shot Multimodal LLMs for ECG Image Classification: A Comparative Study

model-releasesarxiv-cs-lg

arXiv:2606.22889v1 Announce Type: new Abstract: Multimodal large language models (LLMs) are increasingly adopted to interpret 12-lead ECG images, though the interpretations often lack validation. Howe

SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning

arXiv:2606.22873v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly deployed in consumer, medical, financial, and enterprise applications. This broad deployment expands the

SPAR: Semantic-Pixel Self-Alignment and Adaptive Routing for Unified Multimodal Models

arXiv:2606.23041v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable success in visual understanding but remain constrained in visual generation due to the

Multimodal Ordinal Modeling of Alzheimer's Disease Severity Using Structural MRI and Clinical Data

arXiv:2606.11794v1 Announce Type: cross Abstract: Neurodegenerative diseases such as Alzheimer's disease (AD) require accurate and scalable tools for assessing disease severity, yet current clinical s

11 Jun 2026

When to Align, When to Predict: A Phase Diagram for Multimodal Learning

arXiv:2606.11190v1 Announce Type: new Abstract: Cross-modal alignment (CA) and cross-modal prediction (CP) are the dominant paradigms for multimodal representation learning, yet there is no systematic

safetyarxiv-cs-lg

10 Jun 2026

DifferSeg: Towards Diverse Multimodal Binary Segmentation via Differential Perception and Frequency Guidance

arXiv:2606.08906v1 Announce Type: new Abstract: In many binary segmentation tasks, most multimodal methods rely on fixed feature concatenation for cross-modal interaction and straightforward decoder d

9 Jun 2026

Multimodal Generative Engine Optimization: Rank Manipulation for Vision-Language Model Rankers

arXiv:2601.12263v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) integrate visual and textual knowledge into unified representations that increasingly underpin modern retrieval

9 Jun 2026

NeuroAlign: Hierarchical Multimodal Fusion of Dynamic and Structural Neuroimaging for MCI Analysis

arXiv:2606.07635v1 Announce Type: cross Abstract: Multimodal neuroimaging fusion of functional MRI (fMRI) and diffusion tensor imaging (DTI) provides complementary information for cognitive impairment

9 Jun 2026

WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark

arXiv:2606.06538v1 Announce Type: new Abstract: In real-world applications, models are expected to perform reliably across diverse settings. Yet, many existing multimodal benchmarks expand task types

8 Jun 2026

Applications

PAMF: Prior-Aware Multimodal Fusion for Incomplete Time Series Data

arXiv:2606.06328v1 Announce Type: cross Abstract: In healthcare, multimodal time series tasks often operate on incomplete observations in practice, for example when ECG segments are lost because elect

applicationsarxiv-cs-ai

6 Jun 2026

Tutorials

TRACE: A Temporal Conditional Estimation for Multimodal Time Series Foundation Models

arXiv:2606.06285v1 Announce Type: new Abstract: Time series foundation models (TS-FMs) aim to learn generalizable temporal representations that can be adapted to a wide range of downstream tasks. In r

tutorialsarxiv-cs-ai

6 Jun 2026

Overview of the EReL@MIR 2025 Multimodal Document Retrieval Challenge (Track 1)

arXiv:2606.04240v1 Announce Type: cross Abstract: Retrieval over visually-rich documents, pages that interleave text with figures, tables, and charts, is essential for multimodal retrieval-augmented g

4 Jun 2026

TalkPlayData 2: An Agentic Synthetic Data Pipeline for Multimodal Conversational Music Recommendation

arXiv:2509.09685v5 Announce Type: replace-cross Abstract: We present TalkPlayData 2, a synthetic dataset for multimodal conversational music recommendation generated by an agentic data pipeline. In th

3 Jun 2026

Feature Alignment Determines Fusion Strategy: A Comparative Study of Cross-Attention and Concatenation in Multimodal Learning

arXiv:2606.01207v1 Announce Type: new Abstract: The choice between cross-attention and concatenation for multimodal fusion remains governed by practitioner intuition rather than principled understandi

FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning

arXiv:2604.03893v2 Announce Type: replace Abstract: Current multimodal benchmarks for scientific reasoning primarily evaluate local information extraction -- models recognize symbols and values and th

Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling

arXiv:2606.02578v1 Announce Type: cross Abstract: Recent multimodal large language models have demonstrated strong reasoning ability, yet their reliability as automated evaluators remains limited by a

Multimodal Approaches for Visually-Rich Document Type Classification: A Comparative Analysis

arXiv:2606.02162v1 Announce Type: cross Abstract: Document type classification in visually rich documents remains challenging, as relevant information is distributed across textual, visual, and layout

Multimodal Music Recommendation System using LLMs

arXiv:2606.00125v1 Announce Type: cross Abstract: Music recommendation systems typically treat songs as opaque tokens, relying on collaborative interaction histories which overlooks semantic or acoust

PUMA: Layer-Pruned Language Model for Efficient Unified Multimodal Retrieval with Modality-Adaptive Learning

arXiv:2507.08064v3 Announce Type: replace-cross Abstract: As multimedia content expands, the demand for unified multimodal retrieval (UMR) in real-world applications increases. Recent work leverages m

Task-Focused Memorization for Multimodal Agents

arXiv:2605.31075v1 Announce Type: new Abstract: Long-term memory is essential for multimodal agents to build coherent experience, accumulate world knowledge, and achieve continual learning. However, c

1 Jun 2026

OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfields

arXiv:2605.29833v1 Announce Type: new Abstract: As multimodal language models play an increasingly important role in scientific research, materials science offers a critical testbed due to its interdi

29 May 2026

Bounded-Compute Multimodal Regression for Product-Rating Prediction

arXiv:2605.27737v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly attractive for multimodal quality assessment, but their default reliance on autoregressive text generatio

Learning Deliberately, Acting Intuitively: Unlocking Test-Time Reasoning in Multimodal LLMs

arXiv:2507.06999v2 Announce Type: replace-cross Abstract: Reasoning is essential for large language models (LLMs), especially in complex tasks such as mathematical problem solving. However, multimodal

safetyarxiv-cs-cl

Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning

arXiv:2605.28160v1 Announce Type: new Abstract: Existing multimodal reasoning approaches predominantly follow two paradigms: converting visual inputs into text prior to reasoning, or performing end-to

Applications

OphIn-500K: Curating Web-Scale Visual Instructions for Scaling Ophthalmic Multimodal Large Language Models

arXiv:2605.27916v1 Announce Type: cross Abstract: The advancement of general medical Multimodal Large Language Models (MLLMs) has shown great potential for building conversational assistants to suppor

applicationsarxiv-cs-cl

Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini

arXiv:2605.27295v1 Announce Type: new Abstract: We introduce Gemini Embedding 2, a native multimodal embedding model that allows embedding video, audio, image, and text modalities in a unified represe

LaRe: Latent Refocusing for Multimodal Reasoning

model-releasesarxiv-cs-cl

arXiv:2511.02360v4 Announce Type: replace-cross Abstract: Chain of Thought (CoT) reasoning enhances logical performance by decomposing complex tasks, yet its multimodal extension faces a trade-off. Th

Once-For-All: A Train-Once and Select-Anytime Framework for Multimodal Instruction Tuning

arXiv:2605.26761v1 Announce Type: new Abstract: Multimodal instruction tuning is the de facto recipe for adapting vision language models (VLMs), yet instruction data are highly redundant, making data

Re-M3Dr: Rebalanced MultiModal Mean Deviation Regression

arXiv:2605.26513v1 Announce Type: new Abstract: Mean Deviation (MD) is a critical metric for assessing visual field loss in ophthalmology. While previous work has focused solely on predicting MD from

Document Classification Pattern Recognition via Information Fusion: A Systematic Review of Multimodal and Multiview Representation Approaches

arXiv:2605.23910v1 Announce Type: cross Abstract: Information fusion is used widely to improve document classification by the integration of multiple data sources (multimodal) or representations (mult

Is GPT-4o mini Blinded by its Own Safety Filters? Exposing the Multimodal-to-Unimodal Bottleneck in Hate Speech Detection

arXiv:2509.13608v2 Announce Type: replace Abstract: As Large Multimodal Models (LMMs) become integral to daily digital life, understanding their safety architectures is a critical problem for AI Align

safetyarxiv-cs-lg

Multimodality Stacking with Blockwise missing values and application to the PIONeeR biomarkers study for prediction of resistance to immunotherapy

arXiv:2605.25050v1 Announce Type: cross Abstract: Integrating multimodal datasets in clinical oncology is frequently hindered by high dimensionality and blockwise missingness, where entire data source

researcharxiv-cs-lg

Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models

model-releasesarxiv-cs-lg

arXiv:2605.25820v1 Announce Type: new Abstract: Diffusion-based multimodal large language models (dMLLMs) decode by iteratively predicting tokens at multiple masked positions in parallel. This turns e

Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding

arXiv:2605.21852v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in general video understanding, their capacity to interpret invo

22 May 2026

Agents

Beyond Words: Multimodal LLM Knows When to Speak

arXiv:2505.14654v2 Announce Type: replace-cross Abstract: Chatbots via large language models (LLMs) generate fluent responses but often struggle with when to speak, especially for brief, timely listen

agentsarxiv-cs-cl

21 May 2026

Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysis

arXiv:2508.04999v2 Announce Type: replace Abstract: Multimodal sentiment analysis (MSA) aims to understand human emotions by integrating information from multiple modalities, such as text, audio, and

safetyarxiv-cs-lg

21 May 2026

CodeBind: Decoupled Representation Learning for Multimodal Alignment with Unified Compositional Codebook

arXiv:2605.18257v1 Announce Type: cross Abstract: Multimodal representation alignment is pivotal for large language models and robotics. Traditional methods are often hindered by cross-modal informati

Learning Relative Representations for Fine-Grained Multimodal Alignment with Limited Data

arXiv:2605.16834v1 Announce Type: cross Abstract: Multimodal pre-training demonstrates strong generalization performance, but this paradigm is often impractical in domains where paired data are scarce

Agents

Learning to Learn from Multimodal Experience

arXiv:2605.16857v1 Announce Type: new Abstract: Experience-driven learning has emerged as a promising paradigm for enabling agents to improve from interaction trajectories by accumulating and reusing

agentsarxiv-cs-ai

Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages

arXiv:2605.17152v1 Announce Type: new Abstract: Multimodal LLMs are evolving from vision-language to tri-modality that see, hear, and read, yet pipelines and benchmarks remain English-centric and comp

safetyarxiv-cs-cl

Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generation

arXiv:2605.17488v1 Announce Type: new Abstract: The landscape of joint audio and video generation has been fundamentally transformed by the advent of powerful foundation models. Despite these strides,

Semantic Generative Tuning for Unified Multimodal Models

arXiv:2605.18714v1 Announce Type: cross Abstract: Unified multimodal models (UMMs) strive to consolidate visual understanding and visual generation within a single architecture. However, prevailing tr

TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens

arXiv:2605.16638v1 Announce Type: new Abstract: Recent research has demonstrated that Universal Multimodal Embedding (UME) benefits significantly from Chain-of-Thought (CoT) reasoning. In this paradig

Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models

arXiv:2605.15792v1 Announce Type: new Abstract: The long-standing goal of multimodal AI is to build unified models in which visual understanding and visual generation mutually enhance one another. Des