Search: “multimodal”

Modular Multimodal Classification Without Fine-Tuning: A Simple Compositional Approach

arXiv:2605.20674v1 Announce Type: new Abstract: We introduce CoMET, extit{extbf{C}omposing extbf{M}odality extbf{E}ncoders with extbf{T}abular foundation models}, a simple yet highly competitive metho

researcharxiv-cs-lg

21 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Prompt Reinjection: Alleviating Prompt Forgetting in Multimodal Diffusion Transformers

DGX agent

arXiv:2602.06886v3 Announce Type: replace Abstract: Multimodal Diffusion Transformers (MMDiTs) for text-to-image generation maintain separate text and image branches, with bidirectional information fl

researcharxiv-cs-cv

21 May 2026

streaming from modern agents is pretty complex! especially with - parallel tools / subagents - multimodal content - human in the loop events…

streaming from modern agents is pretty complex! especially with - parallel tools / subagents - multimodal content - human in the loop events our new streaming primitives make all of this ergonomic! Ag

agentsharrison-chase--x

21 May 2026

A Nash Equilibrium Framework For Training-Free Multimodal Step Verification

arXiv:2605.20033v1 Announce Type: new Abstract: Multimodal large language models often generate reasoning chains containing subtle errors that lead to incorrect answers. Current verification approache

AffectAI-Capture: A Reproducible Multimodal Protocol for Small-Group Meeting Research

arXiv:2605.19794v1 Announce Type: cross Abstract: We present AffectAI-Capture, a protocol for collecting synchronized multimodal data in four-person meeting-like interactions, combining eye tracking,

Are Tools Always Beneficial? Learning to Invoke Tools Adaptively for Dual-Mode Multimodal LLM Reasoning

arXiv:2605.19852v1 Announce Type: new Abstract: Tool-augmented reasoning has emerged as a promising direction for enhancing the reasoning capabilities of multimodal large language models (MLLMs). Howe

ClinSeekAgent: Automating Multimodal Evidence Seeking for Agentic Clinical Reasoning

arXiv:2605.20176v1 Announce Type: new Abstract: Large language models (LLMs) and agentic systems have shown promise for clinical decision support, but existing works largely assume that evidence has a

DMN: A Compositional Framework for Jailbreaking Multimodal LLMs with Multi-Image Inputs

model-releasesarxiv-cs-ai

arXiv:2605.18915v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are vulnerable to jailbreak attacks, which can elicit harmful responses from MLLMs. Many MLLMs support multi-

Robust Checkpoint Selection for Multimodal LLMs via Agentic Evaluation and Stability-Aware Ranking

arXiv:2605.18852v1 Announce Type: cross Abstract: Checkpoint selection for multimodal large language models (MLLMs) presents significant challenges when performance differentials are marginal and eval

agentsarxiv-cs-ai

Tutorials

An Interpretable Closed-Loop Intelligent Tutoring System for Multimodal Affective Feedback in Asynchronous Presentation Training

arXiv:2605.17468v1 Announce Type: cross Abstract: This paper presents an interpretable closed-loop Intelligent Tutoring System (ITS) that supports feedback-guided practice for developing on-camera ora

tutorialsarxiv-cs-ai

Are Multimodal LLMs Ready for Surveillance? A Reality Check on Zero-Shot Anomaly Detection in the Wild

arXiv:2603.04727v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have demonstrated impressive general competence in video understanding, yet their reliability for rea

safetyarxiv-cs-ai

Auditing Multimodal LLM Raters: Central Tendency Bias in Clinical Ordinal Scoring

arXiv:2605.16386v1 Announce Type: new Abstract: Multimodal large language models (LLMs) are increasingly explored as automated evaluators in clinical settings, yet their scoring behavior on ordinal cl

Beyond Patches: Global-aware Autoregressive Model for Multimodal Few-Shot Font Generation

arXiv:2601.01593v2 Announce Type: replace Abstract: Manual font design is an intricate process that transforms a stylistic visual concept into a coherent glyph set. This challenge persists in automate

Beyond Superficial Unlearning: Sharpness-Aware Robust Erasure of Hallucinations in Multimodal LLMs

model-releasesarxiv-cs-ai

arXiv:2601.16527v2 Announce Type: replace-cross Abstract: Multimodal LLMs are powerful but prone to object hallucinations, which describe non-existent entities and harm reliability. While recent unlea

Controlling Decision Drift in Multimodal Sentiment Analysis with Missing Modalities

arXiv:2605.16889v1 Announce Type: new Abstract: Multimodal sentiment analysis relies on textual, acoustic, and visual signals, yet real-world data often suffer from modality missing and quality imbala

safetyarxiv-cs-cv

Cultivating Forensic Reasoning for Generalizable Multimodal Manipulation Detection

arXiv:2603.01993v2 Announce Type: replace Abstract: Recent advances in generative AI have significantly enhanced the realism of multimodal media manipulation, thereby posing substantial challenges to

GeoSym127K: Scalable Symbolically-verifiable Synthesis for Multimodal Geometric Reasoning

arXiv:2605.16371v1 Announce Type: cross Abstract: Large Multimodal Models (LMMs) often struggle with geometric reasoning due to visual hallucinations and a lack of mathematically precise Chain-of-Thou

Leveraging Multimodal Self-Consistency Reasoning in Coding Motivational Interviewing for Alcohol Use Reduction

arXiv:2605.12987v2 Announce Type: replace Abstract: BACKGROUND: Coding Motivational Interviewing (MI) sessions is essential for understanding client behaviors and predicting outcomes, but it requires

researcharxiv-cs-cl

Reference anything: Gemini Omni extends Gemini's native multimodality, allowing you to blend combinations of text, audio, image, and video i…

model-releasesgoogle-ai--x

Gemini Omni is an extension of Google's Gemini model that enhances its multimodal capabilities by enabling seamless integration of text, audio, image, and video inputs and outputs. This advancement al

Seeing Together:Multi-Robot Cooperative Egocentric Spatial Reasoning with Multimodal Large Language Models

arXiv:2605.18431v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have made substantial progress in egocentric video understanding, but their ability to reason cooperatively fro

Local Ai

Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation

arXiv:2605.18740v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) still struggle with fine-grained visual understanding, where answers often depend on small but decisive evide

local-aiarxiv-cs-ai

Attribute-Grounded Selective Reasoning for Artwork Emotion Understanding with Multimodal Large Language Models

arXiv:2605.15755v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can produce fluent artwork emotion explanations, but they often suffer from attribute flooding: they enumerate

agentsarxiv-cs-cv

18 May 2026

Industry

Introducing Starchild-1 from @odysseyml, the first ever real-time multimodal world model. This a model that can generate interactive simulat…

Introducing Starchild-1 from @odysseyml, the first ever real-time multimodal world model. This a model that can generate interactive simulations of the world that you can—for the first time ever—hear.

industryemad-mostaque--x

18 May 2026

A Mutual Information Lower Bound for Multimodal Regression Active Learning

arXiv:2605.14917v1 Announce Type: new Abstract: Active learning for continuous regression has lacked an acquisition function that targets epistemic uncertainty when the predictive distribution is mult

researcharxiv-cs-lg

AIM-DDI: A Model-Agnostic Multimodal Integration Module for Drug-Drug Interaction Prediction

arXiv:2605.14327v1 Announce Type: cross Abstract: Drug-drug interaction (DDI) prediction is a critical task in computational biomedicine, as adverse interactions between co-administered drugs can caus

Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasoners

arXiv:2605.14709v1 Announce Type: new Abstract: Recent unified models integrate multimodal understanding and generation within a single framework. However, an 'understanding-generation gap' persists,

Diagnosing and Correcting Concept Omission in Multimodal Diffusion Transformers

arXiv:2605.14270v1 Announce Type: new Abstract: Multimodal Diffusion Transformers (MM-DiTs) have achieved remarkable progress in text-to-image generation, yet they frequently suffer from concept omiss

Octopus: History-Free Gradient Orthogonalization for Continual Learning in Multimodal Large Language Models

arXiv:2605.14938v1 Announce Type: cross Abstract: Continual learning in multimodal large language models (MLLMs) aims to sequentially acquire knowledge while mitigating catastrophic forgetting, yet ex

Three new open-source models just landed in ComfyUI natively: → Gemma 4 (Google DeepMind) - multimodal LLM handling text, image, audio, and …

Three new open-source models just landed in ComfyUI natively: → Gemma 4 (Google DeepMind) - multimodal LLM handling text, image, audio, and video input with built-in step-by-step reasoning mode → VOID

model-releasescomfyui--x

Is a Picture Worth a Thousand Words? Adaptive Multimodal Fact-Checking with Visual Evidence Necessity

arXiv:2604.04692v2 Announce Type: replace-cross Abstract: Automated fact-checking is a crucial task that supports a responsible information ecosystem. While recent research has progressed from text-on

LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models

arXiv:2505.15616v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved significant advances in integrating visual and linguistic information, yet their ability to r

Machine Learning-Driven Multimodal Spectroscopic Liquid Biopsy for Early Multicancer Detection

arXiv:2605.13218v1 Announce Type: new Abstract: Cancer is one of the leading causes of death worldwide, making the development of rapid, minimally invasive, label-free and scalable diagnostic strategi

researcharxiv-cs-lg

Seg-Agent: Test-Time Multimodal Reasoning for Training-Free Language-Guided Segmentation

model-releasesarxiv-cs-ai

arXiv:2605.12953v1 Announce Type: cross Abstract: Language-guided segmentation transcends the scope limitations of traditional semantic segmentation, enabling models to segment arbitrary target region

Applications

Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneity

arXiv:2605.12584v1 Announce Type: cross Abstract: Recently, multimodal graph learning (MGL) has garnered significant attention for integrating diverse modality information and structured context to su

applicationsarxiv-cs-ai

ViDR: Grounding Multimodal Deep Research Reports in Source Visual Evidence

arXiv:2605.13034v1 Announce Type: new Abstract: Recent deep research systems have improved the ability of large language models to produce long, grounded reports through iterative retrieval and reason

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

arXiv:2605.12374v1 Announce Type: new Abstract: Visual latent reasoning lets a multimodal large language model (MLLM) create intermediate visual evidence as continuous tokens, avoiding external tools

safetyarxiv-cs-cv

G^2TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models

arXiv:2605.12309v1 Announce Type: new Abstract: The development of separate-encoder Unified multimodal models (UMMs) comes with a rapidly growing inference cost due to dense visual token processing. I

Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Prediction

arXiv:2305.12678v3 Announce Type: replace Abstract: Multimodal Review Helpfulness Prediction (MRHP) aims to rank product reviews based on predicted helpfulness scores and has been widely applied in e-

Local Ai

Instruction Lens Score: Your Instruction Contributes a Powerful Object Hallucination Detector for Multimodal Large Language Models

arXiv:2605.12258v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have achieved remarkable progress, yet the object hallucination remains a critical challenge for reliable deplo

local-aiarxiv-cs-lg

Modality-Inconsistent Continual Learning of Multimodal Large Language Models

arXiv:2412.13050v2 Announce Type: replace-cross Abstract: In this paper, we introduce Modality-Inconsistent Continual Learning (MICL), a new continual learning scenario for Multimodal Large Language M

Multimodal Abstractive Summarization of Instructional Videos with Vision-Language Models

arXiv:2605.11959v1 Announce Type: cross Abstract: Multimodal video summarization requires visual features that align semantically with language generation. Traditional approaches rely on CNN features

safetyarxiv-cs-cl

SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture

arXiv:2605.12500v1 Announce Type: new Abstract: Recent large vision-language models (VLMs) remain fundamentally constrained by a persistent dichotomy: understanding and generation are treated as disti

agentsarxiv-cs-cv

Applications

UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs

arXiv:2605.11856v1 Announce Type: cross Abstract: Multimodal large language models are increasingly expected to perform thinking with images, yet existing visual latent reasoning methods still rely on

applicationsarxiv-cs-cl

An Empirical Analysis of Calibration and Selective Prediction in Multimodal Clinical Condition Classification

arXiv:2603.02719v2 Announce Type: replace Abstract: As artificial intelligence systems move toward clinical deployment, ensuring reliable prediction behavior is fundamental for safety-critical decisio

safetyarxiv-cs-lg

Do multimodal models imagine electric sheep?

arXiv:2605.09693v1 Announce Type: cross Abstract: Yes. We find that large multimodal models develop mental imagery when solving spatial puzzles, and they do imagine sheep when solving sheep puzzles. W

Multimodal Emotion Recognition via Causal-Diffusion Bridge (Affect-Diff)

arXiv:2605.08252v1 Announce Type: new Abstract: Multimodal emotion recognition on CMU-MOSEI faces an extreme imbalance as Happy accounts for 65.9% of samples while three Ekman categories collectively

Personal Visual Context Learning in Large Multimodal Models

arXiv:2605.10936v1 Announce Type: new Abstract: As wearable devices like smart glasses integrate Large Multimodal Models (LMMs) into the continuous first-person visual streams of individual users, the

SalesSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators

arXiv:2605.08334v1 Announce Type: new Abstract: We present SalesSim, a framework and testbed for evaluating the ability of Multimodal Large Language Models (MLLMs) to simulate realistic, persona-drive

safetyarxiv-cs-cl