Search: “multimodal”

BigMac: Breaking the Pareto Frontier of Compute and Memory in Multimodal LLM Training

arXiv:2605.25451v1 Announce Type: new Abstract: Training multimodal large language models (MLLMs) is challenged by both model and data heterogeneity. Existing systems redesign the training pipeline to

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

ConceptM^3oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology

DGX agent

arXiv:2605.24399v1 Announce Type: new Abstract: Healthcare models are transitioning from unimodal prediction toward multimodal reasoning over heterogeneous diagnostic inputs. In computational patholog

applicationsarxiv-cs-ai

26 May 2026

Multimodal Functional Maximum Correlation for Emotion Recognition

arXiv:2512.23076v2 Announce Type: replace-cross Abstract: Emotional states manifest as coordinated yet heterogeneous physiological responses across central and autonomic systems, posing a fundamental

Tutorials

MultiPUFFIN: A Multimodal Domain-Constrained Foundation Model for Molecular Property Prediction of Small Molecules

arXiv:2603.00857v2 Announce Type: replace-cross Abstract: MultiPUFFIN is a domain-informed multimodal foundation model for predicting thermophysical properties of small molecules, addressing a critica

tutorialsarxiv-cs-ai

Prism: A Plug-in Reproducible Infrastructure for Scalable Multimodal Continual Instruction Tuning

arXiv:2605.26110v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) achieve versatility by reformulating diverse tasks into a unified instruction-following framework via instruc

applicationsarxiv-cs-cl

Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs

arXiv:2603.09095v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) can process text presented as images, yet they often perform worse than when the same content is provided a

safetyarxiv-cs-cl

RecGOAT: Graph Optimal Adaptive Transport for LLM-Enhanced Multimodal Recommendation with Dual Semantic Alignment

arXiv:2602.00682v2 Announce Type: replace-cross Abstract: Integrating large language model (LLM) representations into multimodal recommendation has shown promise, yet a fundamental challenge remains l

Multimodal Distribution Matching for Vision-Language Dataset Distillation

arXiv:2605.23482v1 Announce Type: cross Abstract: Dataset distillation compresses large training sets into compact synthetic datasets while preserving downstream performance. As modern systems increas

25 May 2026

Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis

arXiv:2510.08759v2 Announce Type: replace Abstract: Understanding the capability bottlenecks of embodied multimodal large language models (MLLMs) is crucial for improving embodied agents. However, exi

22 May 2026

Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention

arXiv:2605.22072v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a promising paradigm for advancing complex reasoning in large language models, and

researcharxiv-cs-cl

22 May 2026

X-OmniClaw Technical Report: A Unified Mobile Agent for Multimodal Understanding and Interaction

arXiv:2605.05765v2 Announce Type: replace Abstract: Inspired by the development of OpenClaw, there is a growing demand for mobile-based personal agents capable of handling complex and intuitive intera

22 May 2026

Chronicle: A Multimodal Foundation Model for Joint Language and Time Series Understanding

arXiv:2605.20268v1 Announce Type: cross Abstract: Real-world time series come with text: metadata, descriptions, news, reports. Yet time series foundation models process numerical sequences in isolati

iReasoner: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Models

arXiv:2601.05877v3 Announce Type: replace Abstract: Recent work shows that large multimodal models (LMMs) can self-improve from unlabeled data via self-play and intrinsic feedback. Yet existing self-e

researcharxiv-cs-cl

JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA

arXiv:2605.20284v1 Announce Type: new Abstract: Industrial anomaly detection has been significantly advanced by Large Multimodal Models (LMMs), enabling diverse human instructions beyond detection, pa

Latent Space Guided Scenario Sampling for Multimodal Segmentation Under Missing Modalities

arXiv:2605.20372v1 Announce Type: new Abstract: Multimodal semantic segmentation benefits remote sensing analysis by combining complementary information from different sensor modalities. In real-world

applicationsarxiv-cs-cv

Local-sensitive connectivity filter (ls-cf): A post-processing unsupervised improvement of the frangi, hessian and vesselness filters for multimodal vessel segmentation

arXiv:2605.21251v1 Announce Type: cross Abstract: A retinal vessel analysis is a procedure that can be used as an assessment of risks to the eye. This work proposes an unsupervised multimodal approach

researcharxiv-cs-cv

Agents

ProCrit: Self-Elicited Multi-Perspective Reasoning with Critic-Guided Revision for Multimodal Sarcasm Detection

arXiv:2605.20867v1 Announce Type: cross Abstract: Multimodal sarcasm detection requires reasoning over cross-modal incongruities between literal expression and intended meaning, yet the specific analy

agentsarxiv-cs-cv

ProtoPathway: Biologically Structured Prototype-Pathway Fusion for Multimodal Cancer Survival Prediction

arXiv:2605.21454v1 Announce Type: new Abstract: We introduce ProtoPathway, an interpretable-by-design multimodal framework for cancer survival prediction that unifies whole slide imaging and transcrip

researcharxiv-cs-cv

Hallucination as Exploit: Evidence-Carrying Multimodal Agents

model-releasesarxiv-cs-ai

arXiv:2605.19192v1 Announce Type: new Abstract: Multimodal agents use screenshots, documents, and webpages to choose tool calls. When a false visual claim triggers a click, email, extraction, or trans

Learning ORDER-Aware Multimodal Representations for Composite Materials Design

arXiv:2602.02513v2 Announce Type: replace Abstract: Artificial intelligence has shown remarkable success in materials discovery and property prediction, particularly for crystalline and polymer system

safetyarxiv-cs-lg

Local Ai

Slot-MLLM: Object-Centric Visual Tokenization for Multimodal LLM

arXiv:2505.17726v3 Announce Type: replace-cross Abstract: Recently, multimodal large language models (MLLMs) have emerged as a key approach in achieving artificial general intelligence. In particular,

local-aiarxiv-cs-ai

Surviving the Unseen: Predictive Defense for Novel Multi-Turn Multimodal Attacks

arXiv:2605.18988v1 Announce Type: cross Abstract: The expansion of Multimodal Large Language Models (MLLMs) and their integration into autonomous agentic workflows has introduced a non-stationary atta

LatentUMM: Dual Latent Alignment for Unified Multimodal Models

arXiv:2605.17766v1 Announce Type: new Abstract: Unified multimodal models (UMMs) achieve strong performance in both understanding and generation by learning a shared latent space, yet they often exhib

MedMIX: Modality-Internal Expert Fusion for Multimodal Medical Diagnosis

arXiv:2605.16639v1 Announce Type: new Abstract: Multimodal clinical prediction faces three challenges: multiple foundation models (FMs) with complementary strengths per modality, pervasive missing mod

RAVE: Re-Allocating Visual Attention in Large Multimodal Models

arXiv:2605.18359v1 Announce Type: new Abstract: Large multimodal models (LMMs) inherit the self-attention mechanism of pretrained language backbones, yet standard attention can exhibit suboptimal allo

Local Ai

Speech-Guided Multimodal Learning for Vocal Tract Segmentation in Real-Time MRI

arXiv:2605.18466v1 Announce Type: new Abstract: Segmenting vocal tract articulators in real-time MRI (rtMRI) is a challenging dynamic image segmentation problem characterized by low contrast, rapid mo

local-aiarxiv-cs-cv

UTOPYA: A Multimodal Deep Learning Framework for Physics-Informed Anomaly Detection and Time-Series Prediction

model-releasesarxiv-cs-lg

arXiv:2605.18188v1 Announce Type: new Abstract: Anomaly detection in batch processes is hindered by transient dynamics, scarce fault labels, and reliance on single-modality sensor data. This work intr

Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Models

model-releasesarxiv-cs-ai

arXiv:2605.18160v1 Announce Type: cross Abstract: In recent years, multimodal large language models (MLLMs) have achieved remarkable progress, primarily attributed to effective paradigms for integrati

Agents

Agent4POI: Agentic Context-Conditioned Affordance Reasoning for Multimodal Point-of-Interest Recommendation

arXiv:2605.15203v1 Announce Type: cross Abstract: We introduce Agent4POI, the first POI recommendation framework that generates context-conditioned multimodal representations at recommendation time, r

agentsarxiv-cs-ai

18 May 2026

ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models

arXiv:2605.15687v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) may memorize sensitive cross-modal information during pretraining, making machine unlearning (MU) crucial. Ex

18 May 2026

MuteBench: Modality Unavailability Tolerance Evaluation for Incomplete Multimodal Fusion

model-releasesarxiv-cs-lg

arXiv:2605.15235v1 Announce Type: new Abstract: Multimodal physiological data powers clinical AI systems from intensive care units to wearable devices, but sensors routinely fail in practice. Two fail

18 May 2026

FlowInOne:Unifying Multimodal Generation as Image-in, Image-out Flow Matching

arXiv:2604.06757v2 Announce Type: replace Abstract: Multimodal generation has long been dominated by text-driven pipelines where language dictates vision but cannot reason or create within it. We chal

From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG

arXiv:2605.15019v1 Announce Type: new Abstract: Multimodal Retrieval-Augmented Generation (RAG) systems retrieve evidence at coarse granularities (entire images or scenes), creating a mismatch with fi

Agents

MediaClaw: Multimodal Intelligent-Agent Platform Technical Report

arXiv:2605.14771v1 Announce Type: new Abstract: MediaClaw is a multimodal agent platform built on the OpenClaw ecosystem. Its core design follows a three-layer architecture of unified abstraction, plu

agentsarxiv-cs-ai

MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory

arXiv:2605.15128v1 Announce Type: cross Abstract: Long-term agent memory is increasingly multimodal, yet existing evaluations rarely test whether agents preserve the visual evidence needed for later r

REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understanding

arXiv:2511.13026v3 Announce Type: replace Abstract: Self-reflection mechanisms that rely on purely text-based rethinking processes perform well in most multimodal tasks. However, when directly applied

Tutorials

To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Model

arXiv:2605.14291v1 Announce Type: cross Abstract: The rapid advancement of Large Vision-Language Models (LVLMs) is increasingly accompanied by unauthorized scraping and training on multimodal web data

tutorialsarxiv-cs-ai

Uncovering Trajectory and Topological Signatures in Multimodal Pediatric Sleep Embeddings

arXiv:2605.14156v1 Announce Type: new Abstract: While generative models have shown promise in pediatric sleep analysis, the latent structure of their multimodal embeddings remains poorly understood. T

ChatSR: Multimodal Large Language Models for Scientific Formula Discovery

arXiv:2406.05410v3 Announce Type: replace Abstract: Current multimodal large language models (MLLMs) are mainly focused on the understanding and processing of perceptual modalities such as images and

14 May 2026

Exploring Multimodal LMMs for Online Episodic Memory Question Answering on the Edge

arXiv:2602.22455v2 Announce Type: replace Abstract: We investigate the feasibility of using Multimodal Large Language Models (MLLMs) for real-time online episodic memory question answering. While clou

14 May 2026

MILM: Large Language Models for Multimodal Irregular Time Series with Informative Sampling

arXiv:2605.13711v1 Announce Type: new Abstract: Multimodal irregular time series (MITS) consist of asynchronous and irregularly sampled observations from heterogeneous numerical and textual channels.

applicationsarxiv-cs-lg

14 May 2026

AIA: Rethinking Architecture Decoupling Strategy In Unified Multimodal Model

arXiv:2511.22663v5 Announce Type: replace Abstract: Unified multimodal models for image generation and understanding represent a significant step toward AGI and have attracted widespread attention fro

13 May 2026

Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policies

arXiv:2605.11387v1 Announce Type: new Abstract: We address the problem of fine-tuning pre-trained generative policies with reinforcement learning (RL) while preserving the multimodality of their actio

13 May 2026

Calibrated Multimodal Representation Learning with Missing Modalities

arXiv:2511.12034v2 Announce Type: replace Abstract: Multimodal representation learning harmonizes distinct modalities by aligning them into a unified latent space. Recent research generalizes traditio

13 May 2026

Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteria

arXiv:2605.08354v1 Announce Type: new Abstract: Aligning multimodal generative models with human preferences demands reward signals that respect the compositional, multi-dimensional structure of human

Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning

arXiv:2605.10765v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) achieve strong performance through instruction tuning, yet real-world deployment often requires continual cap

applicationsarxiv-cs-ai

EverydayMMQA: A Multilingual and Multimodal Framework for Culturally Grounded Spoken Visual QA

model-releasesarxiv-cs-ai

arXiv:2510.06371v2 Announce Type: replace-cross Abstract: Large-scale multimodal models achieve strong results on tasks like Visual Question Answering (VQA), but they are often limited when queries re

jina-embeddings-v5-omni: Text-Geometry-Preserving Multimodal Embeddings via Frozen-Tower Composition

arXiv:2605.08384v1 Announce Type: new Abstract: In this work, we introduce frozen-encoder model composition, a novel approach to multimodal embedding models. We build on the VLM-style architecture, in