Search: “multimodal”

Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling

arXiv:2606.02578v1 Announce Type: cross Abstract: Recent multimodal large language models have demonstrated strong reasoning ability, yet their reliability as automated evaluators remains limited by a

2 Jun 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Multimodal Approaches for Visually-Rich Document Type Classification: A Comparative Analysis

DGX agent

arXiv:2606.02162v1 Announce Type: cross Abstract: Document type classification in visually rich documents remains challenging, as relevant information is distributed across textual, visual, and layout

model-releasesarxiv-cs-ai

2 Jun 2026

Multimodal Music Recommendation System using LLMs

arXiv:2606.00125v1 Announce Type: cross Abstract: Music recommendation systems typically treat songs as opaque tokens, relying on collaborative interaction histories which overlooks semantic or acoust

2 Jun 2026

PUMA: Layer-Pruned Language Model for Efficient Unified Multimodal Retrieval with Modality-Adaptive Learning

arXiv:2507.08064v3 Announce Type: replace-cross Abstract: As multimedia content expands, the demand for unified multimodal retrieval (UMR) in real-world applications increases. Recent work leverages m

2 Jun 2026

Task-Focused Memorization for Multimodal Agents

arXiv:2605.31075v1 Announce Type: new Abstract: Long-term memory is essential for multimodal agents to build coherent experience, accumulate world knowledge, and achieve continual learning. However, c

1 Jun 2026

OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfields

arXiv:2605.29833v1 Announce Type: new Abstract: As multimodal language models play an increasingly important role in scientific research, materials science offers a critical testbed due to its interdi

29 May 2026

Bounded-Compute Multimodal Regression for Product-Rating Prediction

arXiv:2605.27737v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly attractive for multimodal quality assessment, but their default reliance on autoregressive text generatio

Learning Deliberately, Acting Intuitively: Unlocking Test-Time Reasoning in Multimodal LLMs

arXiv:2507.06999v2 Announce Type: replace-cross Abstract: Reasoning is essential for large language models (LLMs), especially in complex tasks such as mathematical problem solving. However, multimodal

safetyarxiv-cs-cl

Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning

arXiv:2605.28160v1 Announce Type: new Abstract: Existing multimodal reasoning approaches predominantly follow two paradigms: converting visual inputs into text prior to reasoning, or performing end-to

Applications

OphIn-500K: Curating Web-Scale Visual Instructions for Scaling Ophthalmic Multimodal Large Language Models

arXiv:2605.27916v1 Announce Type: cross Abstract: The advancement of general medical Multimodal Large Language Models (MLLMs) has shown great potential for building conversational assistants to suppor

applicationsarxiv-cs-cl

Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini

arXiv:2605.27295v1 Announce Type: new Abstract: We introduce Gemini Embedding 2, a native multimodal embedding model that allows embedding video, audio, image, and text modalities in a unified represe

LaRe: Latent Refocusing for Multimodal Reasoning

model-releasesarxiv-cs-cl

arXiv:2511.02360v4 Announce Type: replace-cross Abstract: Chain of Thought (CoT) reasoning enhances logical performance by decomposing complex tasks, yet its multimodal extension faces a trade-off. Th

Once-For-All: A Train-Once and Select-Anytime Framework for Multimodal Instruction Tuning

arXiv:2605.26761v1 Announce Type: new Abstract: Multimodal instruction tuning is the de facto recipe for adapting vision language models (VLMs), yet instruction data are highly redundant, making data

Re-M3Dr: Rebalanced MultiModal Mean Deviation Regression

arXiv:2605.26513v1 Announce Type: new Abstract: Mean Deviation (MD) is a critical metric for assessing visual field loss in ophthalmology. While previous work has focused solely on predicting MD from

Document Classification Pattern Recognition via Information Fusion: A Systematic Review of Multimodal and Multiview Representation Approaches

arXiv:2605.23910v1 Announce Type: cross Abstract: Information fusion is used widely to improve document classification by the integration of multiple data sources (multimodal) or representations (mult

Is GPT-4o mini Blinded by its Own Safety Filters? Exposing the Multimodal-to-Unimodal Bottleneck in Hate Speech Detection

arXiv:2509.13608v2 Announce Type: replace Abstract: As Large Multimodal Models (LMMs) become integral to daily digital life, understanding their safety architectures is a critical problem for AI Align

safetyarxiv-cs-lg

Multimodality Stacking with Blockwise missing values and application to the PIONeeR biomarkers study for prediction of resistance to immunotherapy

arXiv:2605.25050v1 Announce Type: cross Abstract: Integrating multimodal datasets in clinical oncology is frequently hindered by high dimensionality and blockwise missingness, where entire data source

researcharxiv-cs-lg

Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models

model-releasesarxiv-cs-lg

arXiv:2605.25820v1 Announce Type: new Abstract: Diffusion-based multimodal large language models (dMLLMs) decode by iteratively predicting tokens at multiple masked positions in parallel. This turns e

Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding

arXiv:2605.21852v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in general video understanding, their capacity to interpret invo

22 May 2026

Agents

Beyond Words: Multimodal LLM Knows When to Speak

arXiv:2505.14654v2 Announce Type: replace-cross Abstract: Chatbots via large language models (LLMs) generate fluent responses but often struggle with when to speak, especially for brief, timely listen

agentsarxiv-cs-cl

21 May 2026

Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysis

arXiv:2508.04999v2 Announce Type: replace Abstract: Multimodal sentiment analysis (MSA) aims to understand human emotions by integrating information from multiple modalities, such as text, audio, and

safetyarxiv-cs-lg

21 May 2026

CodeBind: Decoupled Representation Learning for Multimodal Alignment with Unified Compositional Codebook

arXiv:2605.18257v1 Announce Type: cross Abstract: Multimodal representation alignment is pivotal for large language models and robotics. Traditional methods are often hindered by cross-modal informati

Learning Relative Representations for Fine-Grained Multimodal Alignment with Limited Data

arXiv:2605.16834v1 Announce Type: cross Abstract: Multimodal pre-training demonstrates strong generalization performance, but this paradigm is often impractical in domains where paired data are scarce

Agents

Learning to Learn from Multimodal Experience

arXiv:2605.16857v1 Announce Type: new Abstract: Experience-driven learning has emerged as a promising paradigm for enabling agents to improve from interaction trajectories by accumulating and reusing

agentsarxiv-cs-ai

Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages

arXiv:2605.17152v1 Announce Type: new Abstract: Multimodal LLMs are evolving from vision-language to tri-modality that see, hear, and read, yet pipelines and benchmarks remain English-centric and comp

safetyarxiv-cs-cl

Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generation

arXiv:2605.17488v1 Announce Type: new Abstract: The landscape of joint audio and video generation has been fundamentally transformed by the advent of powerful foundation models. Despite these strides,

Semantic Generative Tuning for Unified Multimodal Models

arXiv:2605.18714v1 Announce Type: cross Abstract: Unified multimodal models (UMMs) strive to consolidate visual understanding and visual generation within a single architecture. However, prevailing tr

TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens

arXiv:2605.16638v1 Announce Type: new Abstract: Recent research has demonstrated that Universal Multimodal Embedding (UME) benefits significantly from Chain-of-Thought (CoT) reasoning. In this paradig

Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models

arXiv:2605.15792v1 Announce Type: new Abstract: The long-standing goal of multimodal AI is to build unified models in which visual understanding and visual generation mutually enhance one another. Des

18 May 2026

Attention-Based Multimodal Survival Prediction with Cross-Modal Bilinear Fusion

model-releasesarxiv-cs-lg

arXiv:2605.13897v1 Announce Type: cross Abstract: We propose a novel multimodal deep learning framework for patient-level survival prediction, which integrates whole-slide histology features, RNA-seq

15 May 2026

Utility-Oriented Visual Evidence Selection for Multimodal Retrieval-Augmented Generation

arXiv:2605.13277v1 Announce Type: cross Abstract: Visual evidence selection is a critical component of multimodal retrieval-augmented generation (RAG), yet existing methods typically rely on semantic

14 May 2026

Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictions

model-releasesarxiv-cs-cl

arXiv:2211.03524v2 Announce Type: replace Abstract: Modern Review Helpfulness Prediction systems are dependent upon multiple modalities, typically texts and images. Unfortunately, those contemporary a

13 May 2026

Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training

arXiv:2605.11931v1 Announce Type: new Abstract: Post-training with explicit reasoning traces is common to improve the reasoning capabilities of Multimodal Large Language Models (MLLMs). However, acqui

13 May 2026

GLEAM: A Multimodal Imaging Dataset and HAMM for Glaucoma Classification

arXiv:2603.12800v2 Announce Type: replace-cross Abstract: We propose glaucoma lesion evaluation and analysis with multimodal imaging (GLEAM), the first publicly available tri-modal glaucoma dataset co

12 May 2026

HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents

arXiv:2605.07177v1 Announce Type: cross Abstract: Existing multimodal search agents process target entities sequentially, issuing one tool call per entity and accumulating redundant interaction rounds

MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference

arXiv:2605.05225v2 Announce Type: replace-cross Abstract: Mixture-of-Experts Multimodal Large Language Models (MoE MLLMs) suffer from a significant efficiency bottleneck during Expert Parallelism (EP)

Multimodal Latent Reasoning via Hierarchical Visual Cues Injection

arXiv:2602.05359v2 Announce Type: replace Abstract: The advancement of multimodal large language models (MLLMs) has enabled impressive perception capabilities. However, their reasoning process often r

TRAJGANR: Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations

arXiv:2605.06990v1 Announce Type: new Abstract: Multimodal self-supervised learning (MSSL) has emerged as a key paradigm for pretraining geospatial foundation models. However, existing geospatial MSSL

Physiologically Grounded Driver Behavior Classification: SHAP-Driven Elite Feature Selection and Hybrid Gradient Boosting for Multimodal Physiological Signals

arXiv:2605.05120v1 Announce Type: new Abstract: An interpretable and scalable framework for decoding driving behaviors from multimodal physiological signals is proposed in this study. We utilize multi

researcharxiv-cs-lg

7 May 2026

RetentiveKV: State-Space Memory for Uncertainty-Aware Multimodal KV Cache Eviction

arXiv:2605.04075v1 Announce Type: cross Abstract: Multimodal Large Language Models face severe challenges in computational efficiency and memory consumption due to the substantial expansion of the vis

researcharxiv-cs-cl

7 May 2026

TCM-Serve: Modality-aware Scheduling for Multimodal Large Language Model Inference

arXiv:2603.26498v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) power platforms like ChatGPT, Gemini, and Copilot, enabling richer interactions with text, images, an

7 May 2026

Toward Structural Multimodal Representations: Specialization, Selection, and Sparsification via Mixture-of-Experts

arXiv:2605.03348v1 Announce Type: new Abstract: We propose S3 (Specialization, Selection, Sparsification), a framework that rethinks multimodal learning through a structural perspective. Instead of en

researcharxiv-cs-lg

6 May 2026

VEBench:Benchmarking Large Multimodal Models for Real-World Video Editing

arXiv:2605.03276v1 Announce Type: new Abstract: Real-world video editing demands not only expert knowledge of cinematic techniques but also multimodal reasoning to select, align, and combine footage i

6 May 2026

Agentic Learner with Grow-and-Refine Multimodal Semantic Memory

arXiv:2511.21678v2 Announce Type: replace-cross Abstract: MLLMs exhibit strong reasoning on isolated queries, yet they operate de novo -- solving each problem independently and often repeating the sam

safetyarxiv-cs-lg

Applications

MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalization

arXiv:2605.01967v1 Announce Type: cross Abstract: Deploying multimodal models in real-world scenarios requires generalization to new environments where recording conditions differ from training, a cha

applicationsarxiv-cs-cv

Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Time

arXiv:2605.01766v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have revolutionized the landscape of AI, demonstrating impressive capabilities in tackling complex vision and

Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection

arXiv:2605.01638v1 Announce Type: new Abstract: Multimodal deepfakes are proliferating on social media and threaten authenticity, information integrity, and digital forensics. Existing benchmarks are

On the Effectiveness of Integration Methods for Multimodal Dialogue Response Retrieval

model-releasesarxiv-cs-cl

arXiv:2506.11499v2 Announce Type: replace Abstract: Multimodal chatbots have become one of the major topics for dialogue systems in both research community and industry. Recently, researchers have she