Search: “multimodal”

Attention-Based Multimodal Survival Prediction with Cross-Modal Bilinear Fusion

model-releasesarxiv-cs-lg

arXiv:2605.13897v1 Announce Type: cross Abstract: We propose a novel multimodal deep learning framework for patient-level survival prediction, which integrates whole-slide histology features, RNA-seq

15 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Utility-Oriented Visual Evidence Selection for Multimodal Retrieval-Augmented Generation

DGX agent

arXiv:2605.13277v1 Announce Type: cross Abstract: Visual evidence selection is a critical component of multimodal retrieval-augmented generation (RAG), yet existing methods typically rely on semantic

researcharxiv-cs-ai

14 May 2026

Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictions

arXiv:2211.03524v2 Announce Type: replace Abstract: Modern Review Helpfulness Prediction systems are dependent upon multiple modalities, typically texts and images. Unfortunately, those contemporary a

13 May 2026

Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training

arXiv:2605.11931v1 Announce Type: new Abstract: Post-training with explicit reasoning traces is common to improve the reasoning capabilities of Multimodal Large Language Models (MLLMs). However, acqui

safetyarxiv-cs-cv

13 May 2026

GLEAM: A Multimodal Imaging Dataset and HAMM for Glaucoma Classification

arXiv:2603.12800v2 Announce Type: replace-cross Abstract: We propose glaucoma lesion evaluation and analysis with multimodal imaging (GLEAM), the first publicly available tri-modal glaucoma dataset co

12 May 2026

HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents

arXiv:2605.07177v1 Announce Type: cross Abstract: Existing multimodal search agents process target entities sequentially, issuing one tool call per entity and accumulating redundant interaction rounds

MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference

arXiv:2605.05225v2 Announce Type: replace-cross Abstract: Mixture-of-Experts Multimodal Large Language Models (MoE MLLMs) suffer from a significant efficiency bottleneck during Expert Parallelism (EP)

Multimodal Latent Reasoning via Hierarchical Visual Cues Injection

arXiv:2602.05359v2 Announce Type: replace Abstract: The advancement of multimodal large language models (MLLMs) has enabled impressive perception capabilities. However, their reasoning process often r

TRAJGANR: Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations

arXiv:2605.06990v1 Announce Type: new Abstract: Multimodal self-supervised learning (MSSL) has emerged as a key paradigm for pretraining geospatial foundation models. However, existing geospatial MSSL

safetyarxiv-cs-cv

Physiologically Grounded Driver Behavior Classification: SHAP-Driven Elite Feature Selection and Hybrid Gradient Boosting for Multimodal Physiological Signals

arXiv:2605.05120v1 Announce Type: new Abstract: An interpretable and scalable framework for decoding driving behaviors from multimodal physiological signals is proposed in this study. We utilize multi

researcharxiv-cs-lg

7 May 2026

RetentiveKV: State-Space Memory for Uncertainty-Aware Multimodal KV Cache Eviction

arXiv:2605.04075v1 Announce Type: cross Abstract: Multimodal Large Language Models face severe challenges in computational efficiency and memory consumption due to the substantial expansion of the vis

researcharxiv-cs-cl

7 May 2026

TCM-Serve: Modality-aware Scheduling for Multimodal Large Language Model Inference

arXiv:2603.26498v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) power platforms like ChatGPT, Gemini, and Copilot, enabling richer interactions with text, images, an

7 May 2026

Toward Structural Multimodal Representations: Specialization, Selection, and Sparsification via Mixture-of-Experts

arXiv:2605.03348v1 Announce Type: new Abstract: We propose S3 (Specialization, Selection, Sparsification), a framework that rethinks multimodal learning through a structural perspective. Instead of en

researcharxiv-cs-lg

6 May 2026

VEBench:Benchmarking Large Multimodal Models for Real-World Video Editing

arXiv:2605.03276v1 Announce Type: new Abstract: Real-world video editing demands not only expert knowledge of cinematic techniques but also multimodal reasoning to select, align, and combine footage i

6 May 2026

Agentic Learner with Grow-and-Refine Multimodal Semantic Memory

arXiv:2511.21678v2 Announce Type: replace-cross Abstract: MLLMs exhibit strong reasoning on isolated queries, yet they operate de novo -- solving each problem independently and often repeating the sam

safetyarxiv-cs-lg

MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalization

arXiv:2605.01967v1 Announce Type: cross Abstract: Deploying multimodal models in real-world scenarios requires generalization to new environments where recording conditions differ from training, a cha

applicationsarxiv-cs-cv

Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Time

arXiv:2605.01766v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have revolutionized the landscape of AI, demonstrating impressive capabilities in tackling complex vision and

Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection

arXiv:2605.01638v1 Announce Type: new Abstract: Multimodal deepfakes are proliferating on social media and threaten authenticity, information integrity, and digital forensics. Existing benchmarks are

On the Effectiveness of Integration Methods for Multimodal Dialogue Response Retrieval

arXiv:2506.11499v2 Announce Type: replace Abstract: Multimodal chatbots have become one of the major topics for dialogue systems in both research community and industry. Recently, researchers have she

PC-MNet: Dual-Level Congruity Modeling for Multimodal Sarcasm Detection via Polarity-Modulated Attention

arXiv:2605.02447v1 Announce Type: new Abstract: Multimodal sarcasm detection, which aims to precisely identify pragmatic incongruities between literal text and nonverbal cues, has gained substantial a

UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors

arXiv:2605.00658v1 Announce Type: new Abstract: Recent progress has shown that video diffusion models (VDMs) can be repurposed for diverse multimodal graphics tasks. However, existing methods often tr

safetyarxiv-cs-cv

4 May 2026

PRISM: Pre-alignment via Black-box On-policy Distillation for Multimodal Reinforcement Learning

arXiv:2604.28123v1 Announce Type: cross Abstract: The standard post-training recipe for large multimodal models (LMMs) applies supervised fine-tuning (SFT) on curated demonstrations followed by reinfo

1 May 2026

Contrastive Learning for Multimodal Human Activity Recognition with Limited Labeled Data

arXiv:2604.23281v1 Announce Type: cross Abstract: Human activity recognition serves as the foundation for various emerging applications. In recent years, researchers have used collaborative sensing of

applicationsarxiv-cs-cv

28 Apr 2026

EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Models

arXiv:2502.04424v4 Announce Type: replace-cross Abstract: With the integration of multimodal large language models (MLLMs) into robotic systems and AI applications, embedding emotional intelligence (E

28 Apr 2026

MIMIC: A Generative Multimodal Foundation Model for Biomolecules

arXiv:2604.24506v1 Announce Type: new Abstract: Biological function emerges from coupled constraints across sequence, structure, regulation, evolution, and cellular context, yet most foundation models

28 Apr 2026

Agents

DM^3-Nav: Decentralized Multi-Agent Multimodal Multi-Object Semantic Navigation

arXiv:2604.22014v1 Announce Type: cross Abstract: We present DM^3-Nav, a fully decentralized multi-agent semantic navigation system supporting multimodal open-vocabulary goal specification and multi-o

agentsarxiv-cs-ro

Focus Session: Hardware and Software Techniques for Accelerating Multimodal Foundation Models

arXiv:2604.21952v1 Announce Type: cross Abstract: This work presents a multi-layered methodology for efficiently accelerating multimodal foundation models (MFMs). It combines hardware and software co-

Hardware

Multimodal Neural Operators for Real-Time Biomechanical Modelling of Traumatic Brain Injury

arXiv:2510.03248v3 Announce Type: replace-cross Abstract: Background: Traumatic brain injury modeling requires integrating volumetric neuroimaging, demographic parameters, and acquisition metadata. Fi

hardwarearxiv-cs-ai

V-MAGE: A Game Evaluation Framework for Assessing Vision-Centric Capabilities in Multimodal Large Language Models

arXiv:2504.06148v3 Announce Type: replace Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in visual-text processing. However, existi

Latent Denoising Improves Visual Alignment in Large Multimodal Models

arXiv:2604.21343v1 Announce Type: new Abstract: Large Multimodal Models (LMMs) such as LLaVA are typically trained with an autoregressive language modeling objective, providing only indirect supervisi

24 Apr 2026

Tutorials

S1-VL: Scientific Multimodal Reasoning Model with Thinking-with-Images

arXiv:2604.21409v1 Announce Type: new Abstract: We present S1-VL, a multimodal reasoning model for scientific domains that natively supports two complementary reasoning paradigms: Scientific Reasoning

tutorialsarxiv-cs-cv

24 Apr 2026

MATT-Diff: Multimodal Active Target Tracking by Diffusion Policy

arXiv:2511.11931v2 Announce Type: replace Abstract: This paper proposes MATT-Diff: Multimodal Active Target Tracking by Diffusion Policy, a control policy for active multi-target tracking using a mobi

safetyarxiv-cs-ro

23 Apr 2026

Text Steganography with Dynamic Codebook and Multimodal Large Language Model

arXiv:2604.20269v1 Announce Type: cross Abstract: With the popularity of the large language models (LLMs), text steganography has achieved remarkable performance. However, existing methods still have

23 Apr 2026

How to Teach Large Multimodal Models New Skills

arXiv:2510.08564v2 Announce Type: replace Abstract: How can we teach large multimodal models (LMMs) new skills without erasing prior abilities? We study sequential fine-tuning on five target skills wh

safetyarxiv-cs-ai

22 Apr 2026

Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge

arXiv:2603.11665v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have been widely adopted as MLLM-as-a-Judges due to their strong alignment with human judgment across vario

safetyarxiv-cs-cl

22 Apr 2026

Seeing Candidates at Scale: Multimodal LLMs for Visual Political Communication on Instagram

arXiv:2604.19489v1 Announce Type: new Abstract: This paper presents a computational case study that evaluates the capabilities of specialized machine learning models and emerging multimodal large lang

applicationsarxiv-cs-cv

22 Apr 2026

Beyond Feature Fusion: Contextual Bayesian PEFT for Multimodal Uncertainty Estimation

model-releasesarxiv-cs-lg

arXiv:2604.16615v1 Announce Type: new Abstract: We introduce CoCo-LoRA, a multimodal, uncertainty-aware parameter-efficient fine-tuning method for text prediction tasks accompanied by audio context. E

CFMS: Towards Explainable and Fine-Grained Chinese Multimodal Sarcasm Detection Benchmark

arXiv:2604.16372v1 Announce Type: new Abstract: Multimodal sarcasm detection has recently garnered significant attention. However, existing benchmarks suffer from coarse-grained annotations and limite

Cross-Modal Bayesian Low-Rank Adaptation for Uncertainty-Aware Multimodal Learning

model-releasesarxiv-cs-lg

arXiv:2604.16657v1 Announce Type: new Abstract: Large pre-trained language models are increasingly adapted to downstream tasks using parameter-efficient fine-tuning (PEFT), but existing PEFT methods a

ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection

arXiv:2410.04509v3 Announce Type: replace Abstract: As the field of Multimodal Large Language Models (MLLMs) continues to evolve, their potential to revolutionize artificial intelligence is particular

Tutorials

Learning Invariant Modality Representation for Robust Multimodal Learning from a Causal Inference Perspective

arXiv:2604.18460v1 Announce Type: new Abstract: Multimodal affective computing aims to predict humans' sentiment, emotion, intention, and opinion using language, acoustic, and visual modalities. Howev

tutorialsarxiv-cs-lg

Multiplication in Multimodal LLMs: Computation with Text, Image, and Audio Inputs

arXiv:2604.18203v1 Announce Type: new Abstract: Multimodal LLMs can accurately perceive numerical content across modalities yet fail to perform exact multi-digit multiplication when the identical unde

Penny Wise, Pixel Foolish: Bypassing Price Constraints in Multimodal Agents via Visual Adversarial Perturbations

arXiv:2604.16515v1 Announce Type: new Abstract: The rapid proliferation of Multimodal Large Language Models (MLLMs) has enabled mobile agents to execute high-stakes financial transactions, but their a

RA-RRG: Multimodal Retrieval-Augmented Radiology Report Generation with Key Phrase Extraction

arXiv:2504.07415v2 Announce Type: replace-cross Abstract: Automated radiology report generation (RRG) holds potential to reduce the workload of radiologists, and recent advances in multimodal large la

applicationsarxiv-cs-cl

ReFineVLA: Multimodal Reasoning-Aware Generalist Robotic Policies via Teacher-Guided Fine-Tuning

arXiv:2604.17800v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have gained much attention from the research community thanks to their strength in translating multimodal observat

Using Perspectival Words Is Harder Than Vocabulary Words for Humans and Even More So for Multimodal Language Models

arXiv:2506.00065v2 Announce Type: replace Abstract: Multimodal language models (MLMs) increasingly demonstrate human-like communication, yet their use of everyday perspectival words remains poorly und

researcharxiv-cs-cl

Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs

arXiv:2604.16060v1 Announce Type: cross Abstract: Multimodal Reasoning Models (MRMs) leveraging Chain-of-Thought (CoT) based thinking have revolutionized mathematical and logical problem-solving. Howe

20 Apr 2026

Local Ai

GIST: Multimodal Knowledge Extraction and Spatial Grounding via Intelligent Semantic Topology