Search: “multimodal”

Equivariant Multi-agent Reinforcement Learning for Multimodal Vehicle-to-Infrastructure Systems

arXiv:2604.06914v1 Announce Type: new Abstract: In this paper, we study a vehicle-to-infrastructure (V2I) system where distributed base stations (BSs) acting as road-side units (RSUs) collect multimod

safetyarxiv-cs-lg

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Faithful-First Reasoning, Planning, and Acting for Multimodal LLMs

DGX agent

arXiv:2511.08409v4 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) frequently suffer from unfaithfulness, generating reasoning chains that drift from visual evidence or contr

researcharxiv-cs-ai

10 Apr 2026

Faithful GRPO: Improving Visual Spatial Reasoning in Multimodal Language Models via Constrained Policy Optimization

arXiv:2604.08476v1 Announce Type: new Abstract: Multimodal reasoning models (MRMs) trained with reinforcement learning with verifiable rewards (RLVR) show improved accuracy on visual reasoning benchma

safetyarxiv-cs-cv

GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents

arXiv:2604.07429v1 Announce Type: new Abstract: Towards an embodied generalist for real-world interaction, Multimodal Large Language Model (MLLM) agents still suffer from challenging latency, sparse f

Lost in the Hype: Revealing and Dissecting the Performance Degradation of Medical Multimodal Large Language Models in Image Classification

arXiv:2604.08333v1 Announce Type: new Abstract: The rise of multimodal large language models (MLLMs) has sparked an unprecedented wave of applications in the field of medical imaging analysis. However

Towards Effective Long Video Understanding of Multimodal Large Language Models via One-shot Clip Retrieval

arXiv:2512.08410v2 Announce Type: replace Abstract: Due to excessive memory overhead, most Multimodal Large Language Models (MLLMs) can only process videos of limited frames. In this paper, we propose

Tools

Multimodal Embedding & Reranker Models with Sentence Transformers

The Sentence Transformers v5.4 update introduces first-class multimodal support, enabling the same familiar API to encode and compare texts, images, audio, and videos using both `SentenceTransforme...

toolshugging-face

9 Apr 2026

Beyond Invariance: Factor-wise Selective Treatment of Shifts for Multimodal Learning

arXiv:2610.21403v1 Announce Type: new Abstract: Distribution shifts pose a fundamental challenge to robust multimodal learning. Existing methods typically promote invariance by suppressing environment

researcharxiv-cs-cl

16 Oct 2026

Enhancing Multimodal RLVR with Pairwise Token Contrast and Adaptive Trajectory Confidence

arXiv:2610.21671v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves multimodal reasoning, but group-relative policy optimization broadcasts a single sequence

safetyarxiv-cs-cl

16 Oct 2026

GesBench: When and Where Do Multimodal Language Models Perceive Social Meaning?

arXiv:2610.15129v1 Announce Type: new Abstract: Human social perception is incremental: we infer intent before an utterance is complete, revise our interpretation as new cues arrive, and attend to spe

Learning Navigation from Multimodal Demonstrations with Geometric Candidate Modes

arXiv:2610.20241v1 Announce Type: new Abstract: Offline navigation demonstrations are multimodal, because one observation often allows several valid actions, such as passing an obstacle on either side

safetyarxiv-cs-ro

MARS: Harmonizing Multimodal Convergence via Adaptive Rank Search

model-releasesarxiv-cs-lg

arXiv:2603.00720v2 Announce Type: replace Abstract: Fine-tuning Multimodal Large Language Models (MLLMs) with parameter-efficient methods like Low-Rank Adaptation (LoRA) is crucial for task adaptation

MultiFlow: Vision-Driven Multimodal fMRI Encoding via Stimulus-Informed Source Flow Matching

arXiv:2610.15339v1 Announce Type: new Abstract: Current fMRI encoding models frame prediction as deterministic regression, yet identical stimuli evoke a distribution of neural responses (arising from

researcharxiv-cs-ai

NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning

model-releasesnvidia-developer

NVIDIA’s open‑source Nemotron 3.5 Super VL is a 120 B parameter mixture‑of‑experts multimodal model that activates only 12 B parameters per token, enabling combined reasoning, instruction following, t

Reliability-Aware Multimodal Sentiment Prediction Under Local Modality Missingness

arXiv:2610.20543v1 Announce Type: new Abstract: Multimodal sentiment models are usually evaluated on complete utterances, yet streams often lose a contiguous span of text, audio, or video. We study lo

ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoning

arXiv:2610.17183v1 Announce Type: cross Abstract: Current chemistry benchmarks primarily evaluate outcome prediction or reaction pathway generation, but rarely test whether a model can explain why an

GReFEM: Multimodal LLMs as Zero-Shot Semantic Assistants for Physics-Guided 3D Mesh Refinement

arXiv:2607.08798v2 Announce Type: replace-cross Abstract: Adaptive volumetric finite element meshing traditionally requires iterative PDE solvers or heavily supervised, data-driven surrogates trained

Rethinking Projector Training in Multimodal LLMs

arXiv:2610.18598v1 Announce Type: new Abstract: Multimodal LLMs are typically trained by first aligning a vision encoder with a language model via projector training on large image-caption datasets, f

safetyarxiv-cs-cv

SOVABench: A Vehicle Surveillance Action Retrieval Benchmark for Multimodal Large Language Models

arXiv:2601.04824v3 Announce Type: replace Abstract: Automatic identification of events and recurrent behavior analysis are critical for video surveillance. However, most existing content-based video r

VIEWCRAFT: Evolving What Multimodal Agents See and How They Read It

arXiv:2610.15915v1 Announce Type: new Abstract: Reusable skills improve multimodal agents without updating model weights, but their effectiveness depends on both the evidence presented and the procedu

Visual Tokens on Demand: Visual Budgeting for Scalable and Efficient Multi-Image Reasoning in Multimodal LLMs

arXiv:2610.18858v1 Announce Type: new Abstract: Multimodal large language models have advanced rapidly, but most remain optimized for single-image inputs. Multi-image reasoning is costly because each

EmoGavel: Grounding-Aware Reinforcement Learning with Verifiable Rewards for Multimodal Emotion Preference Learning

model-releasesarxiv-cs-lg

arXiv:2610.13353v1 Announce Type: cross Abstract: Multimodal emotion preference learning uses pairwise comparisons to identify which open-ended emotion descriptions align more closely with human judgm

MotionCoach: Constructing Multimodal Guidance for Video Professional Motion Editing

arXiv:2610.12963v1 Announce Type: cross Abstract: Video editing models achieve strong performance in modifying appearance, backgrounds, styles, and motions, empowered by large-scale training data. Bey

Multimodal Image-based Robotic Assistance and Guidance Environment (MIRAGE): A Novel Unified Framework for Surgical Navigation and Visualization

arXiv:2610.12780v1 Announce Type: new Abstract: This paper presents the computational architecture and quantitative preclinical evaluation of MIRAGE, a multimodal image-guided robotic surgery framewor

researcharxiv-cs-ro

Retrieval-Augmented Multimodal Language Model with Heterogeneous Evidence Integration for Clinical Prognosis

arXiv:2610.13510v1 Announce Type: new Abstract: Clinical prognosis requires integrating structured measurements, physiological waveforms, longitudinal history, and population context. We introduce PRI

Autoregressive Retriever: Improving Query Understanding from Item Feedback for Universal Multimodal Retrieval

arXiv:2610.11666v1 Announce Type: cross Abstract: Universal multimodal retrieval typically encodes a query once and ranks independently indexed items by embedding similarity. This design supports effi

CoPoE: Multimodal Fusion via Decomposable Disease-Coordinate Product-of-Experts for Missing-Modality Alzheimer's Diagnosis

arXiv:2610.11394v1 Announce Type: cross Abstract: Multimodal Alzheimer's disease (AD) diagnosis benefits from integrating heterogeneous clinical, imaging, genomic, and biomarker evidence, but clinical

GeoReform: Reflective Formalization Evolution for Multimodal Geometry Problem Solving

arXiv:2610.12391v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) often struggle to identify and use geometric relations in diagrams. Recent methods address this challenge by co

MissMAC-Bench: Building Solid Benchmark for Missing Modality Issue in Robust Multimodal Affective Computing

arXiv:2602.00811v2 Announce Type: replace Abstract: Current Multimodal Affective Computing (MAC) systems heavily rely on the completeness of multiple modalities to accurately understand human's affect

When History Helps and Hurts: Selective History Use across Multimodal Turns

model-releasesarxiv-cs-cl

arXiv:2610.11948v1 Announce Type: new Abstract: Reliable multimodal interaction depends on selective use of conversational history: an earlier question may remain relevant while its previous answer is

MemoCare: An Interactive Multimodal Mobile System for Automated Cognitive Screening

arXiv:2610.10448v1 Announce Type: cross Abstract: MemoCare is an interactive mobile system for automated multimodal cognitive screening. A React Native application combines spoken responses, temporal

8 Oct 2026

Applications

MoR-MLLM: Mixture of Recursions for Efficient Multimodal Large Language Models

arXiv:2610.08830v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable reasoning capabilities across vision and language tasks. However, their massive co

applicationsarxiv-cs-cv

8 Oct 2026

DIPrune: Task-Aware Token Pruning with Dual Importance for Efficient Multimodal Language Models

arXiv:2610.08341v1 Announce Type: new Abstract: Recent training-free pruning approaches for Multimodal Large Language Models (MLLMs) effectively cut computational overhead by exploiting visual redunda

7 Oct 2026

Nonlocal Hamiltonian Dynamics on Sparse Levy Graphs: Spectral Analysis and Multimodal Sampling

arXiv:2610.06904v1 Announce Type: cross Abstract: We develop a sparse graph method for transporting probability mass toward multimodal target distributions through damped nonlocal Hamiltonian dynamics

researcharxiv-cs-lg

7 Oct 2026

Anatomy-aware Fine-grained Multimodal Fusion for Laryngopharyngeal Cancer T-Staging Prediction Using CT and Radiology Report

arXiv:2610.06837v1 Announce Type: new Abstract: Accurate T-staging is crucial for guiding personalized treatment strategies for laryngopharyngeal cancer. However, current clinical practice relies on i

safetyarxiv-cs-cv

Conditional Trajectory Peaks: Single-Pass Multimodal Policies over Action Chunks

arXiv:2610.06104v1 Announce Type: new Abstract: Multimodal imitation learning requires diverse executable futures under the same observation and consistent behavior across replanning cycles. We presen

safetyarxiv-cs-ro

Agents

DREAM: Dynamic Resolution Assignment For Multimodal Multi-agent Debate

arXiv:2610.05615v1 Announce Type: cross Abstract: Multi-agent debate (MAD) has emerged as an effective paradigm to improve the reasoning capabilities of large language models (LLMs) and is increasingl

agentsarxiv-cs-cv

fMRI-TAMCL: Text-Anchored Supervised Multimodal Contrastive Learning for fMRI-Based Brain Disorder Classification

arXiv:2610.05880v1 Announce Type: new Abstract: Resting-state fMRI is important in the classification of brain disorders, but highly multimodal and exhibits strong multisite heterogeneity. Existing me

Recursive Self-Improvement in Unified Multimodal Models

arXiv:2610.03002v1 Announce Type: new Abstract: Unified multimodal models (UMMs) understand and generate both text and images, which lets a model produce its own training data. Existing self-improveme

researcharxiv-cs-cl

5 Oct 2026

FairSSL: Fair Multimodal Self-Supervised Learning

arXiv:2508.16748v2 Announce Type: replace-cross Abstract: Prevalent multimodal self-supervised learning (SSL) methods rely on the redundancy assumption: that different views share substantial task-rel

Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation

arXiv:2610.02123v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures scale model capacity through sparse computation, routing each token through only a small subset of experts. In th

Applications

ScaffoldM3C: A Multimodal Sequential Monte Carlo Framework for Generative Stable Construction Planning

arXiv:2610.00487v1 Announce Type: cross Abstract: Autonomously constructing physically realizable 3D structures remains a significant challenge due to combinatorial action spaces, interchangeable comp

applicationsarxiv-cs-lg

Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents

arXiv:2610.01892v1 Announce Type: cross Abstract: Multimodal agents commonly generate free-form reasoning before each action. For small models, limited model capacity can result in lengthy reasoning t

researcharxiv-cs-ai

From Charts to Code: A Hierarchical Benchmark for Multimodal Models

arXiv:2510.17932v5 Announce Type: replace-cross Abstract: We introduce Chart2Code, a new benchmark for evaluating the chart understanding and code generation capabilities of large multimodal models (L

1 Oct 2026

Ranking-Aware Prompt Optimization for Multimodal Clinical Diagnosis

arXiv:2609.40361v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) are rapidly advancing clinical diagnosis, yet their adaptation pipelines remain anchored to accuracy-based ob

researcharxiv-cs-cl

1 Oct 2026

ShieldCLIP: Selective Safety Alignment for Harmful Content Mitigation in Multimodal Foundation Models

arXiv:2609.39688v1 Announce Type: cross Abstract: Multimodal encoders such as CLIP underlie many downstream systems, but their web-scale training data embed harmful associations that safety alignment

1 Oct 2026

Calibrate the Decisions That Change the Future: On-Policy Post-Training Quantization for Multimodal Large Language Models

arXiv:2609.36828v1 Announce Type: new Abstract: Post-training quantization (PTQ) lowers deployment cost for multimodal large language models, but calibration typically reconstructs fixed sequences wit

30 Sept 2026

MMSkillRisk: Can Agents Stay Safe When Multimodal Skills Become Traps?

arXiv:2609.35912v1 Announce Type: cross Abstract: Agent skills are shareable packages of procedural instructions, tools, and examples. Multimodal skills additionally include visual references that age