Search: “multimodal”

Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models

arXiv:2607.05716v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong perception and reasoning capabilities. However, most existing models focus on isolated

8 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

ELVA: Exploring Ranking-Driven Universal Multimodal Retrieval

DGX agent

arXiv:2606.20280v2 Announce Type: replace-cross Abstract: Leveraging Multimodal Large Language Models (MLLMs) via contrastive learning has become a mainstream paradigm for improving the performance of

model-releasesarxiv-cs-ai

7 Jul 2026

Local Ai

Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection

arXiv:2607.03817v1 Announce Type: new Abstract: Large Multimodal Models (LMMs) show strong few-shot generalization, but industrial anomaly detection remains difficult because defects are small, input

local-aiarxiv-cs-cv

7 Jul 2026

MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training

arXiv:2602.06285v2 Announce Type: replace Abstract: Recent research in geospatial machine learning has demonstrated that models pretrained with self-supervised learning on Earth observation data can p

7 Jul 2026

Purify then Guide: Rethinking Domain Generalization for Multimodal Face Anti-Spoofing

arXiv:2505.09484v2 Announce Type: replace Abstract: Face Anti-Spoofing (FAS) is essential for the security of facial recognition systems in diverse scenarios such as payment processing and surveillanc

7 Jul 2026

InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories

arXiv:2607.02010v1 Announce Type: new Abstract: Multimodal large language models must adapt to evolving tasks and domains, yet continual improvement under bounded deployment footprint remains difficul

3 Jul 2026

MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering

arXiv:2607.01420v1 Announce Type: cross Abstract: As grounded QA systems are increasingly deployed in AI assistants, accurately attributing generated answers to evidence is critical for user trust and

3 Jul 2026

Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning

arXiv:2607.00461v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) are often constrained by a language-space bottleneck, forcing complex visual reasoning into discrete tokens whi

2 Jul 2026

Rosetta: Composable Native Multimodal Pretraining

arXiv:2607.00293v1 Announce Type: cross Abstract: Achieving true artificial general intelligence requires foundation models capable of integrating new modalities without forgetting prior knowledge. Ho

researcharxiv-cs-cl

2 Jul 2026

XSkill: Continual Learning from Experience and Skills in Multimodal Agents

arXiv:2603.12056v3 Announce Type: replace Abstract: Multimodal agents can now tackle complex reasoning tasks with diverse tools, yet they still suffer from inefficient tool use and inflexible orchestr

2 Jul 2026

AA: A Multi-view Multimodal Dataset for Screen-based Gaze Estimation

arXiv:2606.31211v1 Announce Type: new Abstract: We present AA, a multi-view multimodal dataset for screen-based gaze estimation. The dataset captures synchronized facial observations from eight fixed

Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning

arXiv:2606.31825v1 Announce Type: cross Abstract: Recent multimodal large language models have shown great promise in clinical image reasoning, but existing post-training pipelines remain predominantl

safetyarxiv-cs-ai

Tutorials

Building a Multimodal Dataset of Academic Paper for Keyword Extraction

arXiv:2606.31069v1 Announce Type: new Abstract: Up to this point, keyword extraction task typically relies solely on textual data. Neglecting visual details and audio features from image and audio mod

tutorialsarxiv-cs-cl

Learning to Deny: Action Denial in Multimodal Large Language Models

arXiv:2606.31187v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have rapidly advanced video understanding, achieving strong zero-shot and few-shot recognition across standard

Can MLLMs Critique Like Humans? Evaluating Open-Ended Aesthetic Reasoning in Multimodal Large Language Models

arXiv:2606.29689v1 Announce Type: new Abstract: Open-ended aesthetic critique is a challenge for multimodal large language models (MLLMs): unlike multiple-choice aesthetic benchmarks, it has no single

safetyarxiv-cs-cl

COMPASS: Grounding Composition-Intent Guidance in Unified Multimodal Models

arXiv:2606.28696v1 Announce Type: new Abstract: Composition is a high-level visual intent that governs where subjects are placed and how a scene is organized, yet current unified multimodal models rem

Personalizing MLLMs via Reinforced Multimodal Reference Game

arXiv:2606.28845v1 Announce Type: new Abstract: Personalizing Multimodal Large Language Models (MLLMs) aims to recognize users' unique concepts from visual data and provide personalized responses. Alt

Tutorials

Residual-Guided Expert Specialization for Incomplete Multimodal Learning

arXiv:2606.30355v1 Announce Type: cross Abstract: As real-world prediction systems often face missing modalities at inference, incomplete multimodal learning (IML) remains a practical challenge. While

tutorialsarxiv-cs-ai

Agents

RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources

arXiv:2606.29538v1 Announce Type: cross Abstract: Skills are a useful abstraction for software agents, turning human and agent experience into reusable procedural knowledge. Yet existing skill librari

agentsarxiv-cs-ai

Context-specific Credibility-aware Multimodal Fusion with Conditional Probabilistic Circuits

model-releasesarxiv-cs-lg

arXiv:2603.26629v2 Announce Type: replace Abstract: Multimodal fusion requires integrating information from multiple sources that may conflict depending on context. Existing fusion approaches typicall

29 Jun 2026

From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models

arXiv:2606.26196v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have recently made remarkable progress in unifying vision-language understanding and reasoning, especially fo

hisao{}: A GPU-Native Parallel Optimizer for Multimodal Black-Box Functions via Convergence-Anticonvergence Oscillation

model-releasesarxiv-cs-lg

arXiv:2606.26164v1 Announce Type: new Abstract: Finding all modes of a multimodal black-box function is a fundamental challenge in optimization, Bayesian inference, and scientific computing. Existing

Tutorials

Modeling Local, Global, and Cross-Modal Context in Multimodal 3D MRI

arXiv:2606.26894v1 Announce Type: new Abstract: Brain MRI poses a fundamental challenge for machine learning: models must learn from high-dimensional 3D data spanning multiple co-registered modalities

tutorialsarxiv-cs-cv

Position Rebinding Cache Reuse: Replay-Free Visual Revisiting for Interleaved Multimodal Reasoning

arXiv:2606.26631v1 Announce Type: new Abstract: Interleaved multimodal reasoning improves visual grounding by revisiting visual evidence during multi-step generation, yet existing methods typically re

SyncLoop: A Multimodal Dual-Loop Framework for Self-Improving Mathematical Reasoning

model-releasesarxiv-cs-cl

arXiv:2507.16518v3 Announce Type: replace-cross Abstract: Recent advances in multimodal large language models (MLLMs) have shown impressive reasoning capabilities. However, further enhancing existing

25 Jun 2026

Accelerating Multimodal Large Language Models with Prior-Corrected Token Reduction

arXiv:2606.24156v1 Announce Type: new Abstract: Visual token reduction has emerged as an effective strategy for accelerating Multimodal Large Language Models (MLLMs). Many existing methods prune token

FlowR2A: Learning Reward-to-Action Distribution for Multimodal Driving Planning

arXiv:2606.24231v1 Announce Type: new Abstract: Multimodal driving planning faces a long-standing tension between two paradigms: scoring-based methods benefit from dense reward supervision but are con

safetyarxiv-cs-ai

Mind the Heads: Topological Representation Alignment for Multimodal LLMs

arXiv:2606.23885v1 Announce Type: cross Abstract: Representation alignment has emerged as an effective approach to improve Multimodal Large Language Models (MLLMs) by regularizing their internal repre

Spectral Evolution-Guided Token Pruning in Multimodal Large Language Models

arXiv:2606.24165v1 Announce Type: new Abstract: Reducing visual token redundancy is critical for accelerating Multimodal Large Language Models (MLLMs) without degrading cross-modal reasoning performan

safetyarxiv-cs-cv

A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models

arXiv:2511.15098v2 Announce Type: replace Abstract: Discrete diffusion-based multimodal large language models (dMLLMs) have emerged as a promising alternative to autoregressive MLLMs thanks to their a

Applications

Beyond Time Series: Spatial Reasoning for Epidemic Forecasting via Multimodal Learning

arXiv:2606.22171v1 Announce Type: new Abstract: Epidemic forecasting models typically rely on surveillance data reported over administrative regions, treating them as atomic units, thereby obscuring s

applicationsarxiv-cs-lg

Agents

Dynamic multi-agent deep reinforcement learning-based pricing and incentivization approach in multimodal transportation networks

arXiv:2606.23257v1 Announce Type: new Abstract: In multimodal transportation systems, shared mobility services (SMSs) are promoted for their potential to enhance flexibility and reduce congestion. How

agentsarxiv-cs-lg

Local Ai

EnTrust: Modeling Inter-Modal Conflict for Trustworthy Multimodal Medical Image Analysis

arXiv:2606.21384v1 Announce Type: new Abstract: Multimodal medical imaging fuses complementary anatomical and functional information, yet modalities frequently disagree in pathologically heterogeneous

local-aiarxiv-cs-cv

Extraction and Analysis of Multimodal Concepts in Vision Language Models through Sparse Autoencoders

arXiv:2606.21197v1 Announce Type: new Abstract: Vision Language Models (VLMs) have demonstrated impressive performance in tasks requiring joint understanding of images and text, such as image captioni

safetyarxiv-cs-cv

Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse

arXiv:2606.23581v1 Announce Type: cross Abstract: Multimodal agents repeatedly re-examine the same video frames, UI screenshots, and rendered artifacts as their context window slides and reasoning ite

EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards

arXiv:2511.16672v4 Announce Type: replace Abstract: Recent advances in large multimodal models (LMMs) have enabled impressive reasoning and perception abilities, yet most existing training pipelines s

Human-Guided Agentic AI for Multimodal Clinical Prediction: Lessons from the AgentDS Healthcare Benchmark

arXiv:2602.19502v2 Announce Type: replace Abstract: Agentic AI systems are increasingly capable of autonomous data science workflows, yet clinical prediction tasks demand domain expertise that purely

InternVideo3: Agentify Foundation Models with Multimodal Contextual Reasoning

arXiv:2606.12195v1 Announce Type: new Abstract: Recent progress in foundation models has shifted toward agentic behavior involving multi-step reasoning and tool use. However, open-source efforts large

safetyarxiv-cs-cv

Latent World Recovery for Multimodal Learning with Missing Modalities

arXiv:2606.12362v1 Announce Type: cross Abstract: We study multimodal learning under missing modalities, with particular motivation from bioscience applications in which heterogeneous modalities are o

safetyarxiv-cs-ai

Applications

M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset

arXiv:2510.23508v3 Announce Type: replace Abstract: Existing real-world datasets for multimodal fact-checking have multiple limitations: they contain few instances, cover on only one or two languages,

applicationsarxiv-cs-cl

Local Ai

MultiToP: Learning to Patch Visual Tokens to Mitigate Hallucinations in Video Large Multimodal Models

arXiv:2606.11792v1 Announce Type: cross Abstract: Video Large Multimodal Models have achieved remarkable progress in video understanding, yet they remain prone to hallucinations, where generated respo

local-aiarxiv-cs-ai

Parameter-Efficient Adapter Tuning for Tabular-Image Multimodal Learning

arXiv:2606.11682v1 Announce Type: new Abstract: Tabular-image multimodal learning aims to improve predictive modeling by jointly using structured tabular attributes and visual data. Although pretraine

Agents

ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering

arXiv:2510.04514v3 Announce Type: replace Abstract: Recent multimodal LLMs have shown promise in chart-based visual question answering, but their performance declines sharply on unannotated charts-tho

agentsarxiv-cs-ai

LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts

arXiv:2606.09907v1 Announce Type: cross Abstract: Multimodal clinical learning is increasingly important for integrating diverse patient data, including imaging, text, and personalised health records.

One Token per Multimodal Evidence: Latent Memory for Resource-Constrained QA

arXiv:2606.10572v1 Announce Type: new Abstract: External memory effectively grounds large language models (LLMs) and vision-language models (VLMs)-based question answering (QA) in relevant multimodal

Applications

SynIB: Informational Bottleneck for Maximizing Synergy in Multimodal Learning

arXiv:2606.09853v1 Announce Type: new Abstract: A central objective in multimodal learning is to capture synergy: task-relevant information that arises only from the joint use of multiple modalities,

applicationsarxiv-cs-lg

COMPASS: Complete Multimodal Fusion via Proxy Tokens and Shared Spaces for Ubiquitous Sensing

arXiv:2604.02056v2 Announce Type: replace Abstract: Missing modalities in multimodal sensing cause not only information loss but also a fusion-interface mismatch: a fusion head trained on a canonical

9 Jun 2026

Late-Layer Fusion is Enough: Dual-Path Vision Token Routing for Multimodal Large Language Models under Visual Saturation

arXiv:2606.09131v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) commonly inherit the deep, symmetric Transformer backbone designed for unimodal text modeling, and apply the sa