Search: “multimodal”

Almieyar-Oryx-BloomBench: A Bilingual Multimodal Benchmark for Cognitively Informed Evaluation of Vision-Language Models

arXiv:2606.05531v1 Announce Type: cross Abstract: Despite the rapid progress of Vision-Language Models (VLMs), the field lacks benchmarks that rigorously diagnose their true reasoning abilities and ch

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

DisasterBench: A Multimodal Benchmark for UAV-Based Disaster Response in Complex Environments

DGX agent

arXiv:2606.06217v1 Announce Type: new Abstract: When a disaster unfolds, responders must answer not only what is happening, but also why it is happening, what will happen next, and what to do now, oft

model-releasesarxiv-cs-cv

5 Jun 2026

LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing

model-releasesarxiv-cs-cv

arXiv:2606.06042v1 Announce Type: new Abstract: Developing unified video generation and editing models capable of interpreting interleaved multimodal inputs is a promising yet challenging frontier fie

Agents

MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA

arXiv:2606.05749v1 Announce Type: new Abstract: Iterative retrieval-reasoning agents have recently shown promise for multimodal long-document question answering. However, most existing systems maintai

agentsarxiv-cs-cl

Local Ai

Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Heads

arXiv:2606.05843v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) demonstrate remarkable proficiency on complex vision-language tasks, the mechanisms by which they extract

local-aiarxiv-cs-cl

Seeing is Believing? Evaluating Vision-Language Model Susceptibility in Agent-to-Agent Multimodal Persuasion

arXiv:2510.22768v2 Announce Type: replace Abstract: As autonomous agents increasingly interact, they inevitably attempt to influence one another. While prior work in text-only settings has explored th

BreastGPT: A Multimodal Large Language Model for the Full Spectrum of Breast Cancer Clinical Routine

arXiv:2606.04911v1 Announce Type: cross Abstract: Breast cancer remains a leading cause of cancer-related mortality among women. Its clinical management requires multimodal reasoning across a clinical

4 Jun 2026

FindIt: A Format-Informed Visual Detection Benchmark for Generalist Multimodal LLMs

model-releasesarxiv-cs-cv

arXiv:2606.04282v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are predominantly evaluated on free-form vision-language tasks such as visual question answering, captioning, a

4 Jun 2026

MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A

arXiv:2606.04231v1 Announce Type: cross Abstract: Recent advances in multimodal retrieval-augmented generation (MM-RAG) have shifted toward minimal parsing, relying on page-level images for producing

4 Jun 2026

Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow Matching

arXiv:2602.12221v2 Announce Type: replace Abstract: We propose UniDFlow, a unified discrete flow-matching framework for multimodal understanding, generation, and editing. It decouples understanding an

safetyarxiv-cs-cv

CORE: Conflict-Oriented Reasoning for General Multimodal Manipulation Detection

arXiv:2606.03066v1 Announce Type: new Abstract: The rapid rise of generative AI has made multimodal fake news increasingly realistic and pervasive, posing severe threats to public trust and social sta

MemVerse: Multimodal Memory for Lifelong Learning Agents

arXiv:2512.03627v2 Announce Type: replace Abstract: Despite rapid progress in large-scale language and vision models, AI agents still suffer from a fundamental limitation: they cannot remember. Withou

Social Caption: Evaluating Social Understanding in Multimodal Models

arXiv:2601.14569v2 Announce Type: replace Abstract: Social understanding abilities are crucial for multimodal large language models (MLLMs) to interpret human social interactions. We introduce SOCIAL

researcharxiv-cs-cl

Spectral-Progressive Thought Flow for Lightweight Multimodal Reasoning

arXiv:2606.02842v1 Announce Type: new Abstract: Multimodal spatial reasoning often relies on long chains of intermediate textual and visual thoughts, where accumulating visual tokens and dense cross-m

researcharxiv-cs-lg

Amazing deep dive from the @togethercompute team on serving MiniMax M3 in production. M3 with its 1M context, native multimodality and MiniM…

applicationstogether-ai--x

Amazing deep dive from the @togethercompute team on serving MiniMax M3 in production. M3 with its 1M context, native multimodality and MiniMax Sparse Attention requires real work across paged decode,

CRAM: Centroid-Routing and Adaptive MoE for Multimodal Continual Instruction Tuning

arXiv:2606.02502v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) unify heterogeneous vision-language tasks under a shared generative framework via instruction tuning, yet real-

DraDDP: A Multimodal Multi-Party Dialogue Discourse Parsing Dataset

arXiv:2606.00012v1 Announce Type: cross Abstract: Multi-party dialogue discourse parsing aims to identify dependency structures and relation types between utterances in conversations. Previous studies

Hardware

MiniMax-M3 combines 1M context, native multimodality, and MiniMax Sparse Attention. The next layer is serving it efficiently: KV-block-major…

MiniMax-M3 combines 1M context, native multimodality, and MiniMax Sparse Attention. The next layer is serving it efficiently: KV-block-major sparse attention, paged MSA decode, optimized index scoring

hardwaretogether-ai--x

MLLM-Microscope: Unlocking Hidden Structure Within Multimodal Large Language Models

arXiv:2606.00909v1 Announce Type: cross Abstract: This work presents MLLM-Microscope, a novel system designed for analyzing the hidden representations within Multimodal Large Language Models (MLLMs).

Multimodal Action Diffusion for Robust End-to-End Autonomous Driving

model-releasesarxiv-cs-cv

arXiv:2606.02105v1 Announce Type: new Abstract: End-to-End Autonomous Driving (E2E-AD) systems have largely converged on predicting intermediate trajectory waypoints, delegating final control to hand-

ProtoAda: Prototype-Guided Adaptive Adapter Expansion and Geometric Consolidation for Multimodal Continual Instruction Tuning

arXiv:2606.02576v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) achieve strong performance through instruction tuning, but real-world deployment requires them to continually a

applicationsarxiv-cs-cv

Tools

Serving MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without Regrets

MiniMax-M3 is a large language model capable of handling 1 million token contexts and multimodal inputs while maintaining efficient inference performance. Together AI's blog post discusses techniques

toolstogether-ai-blog

Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition

arXiv:2606.00959v1 Announce Type: new Abstract: Understanding modality interaction in multimodal large language models (MLLMs) is central to reliable deployment. We introduce Partial Information Decom

UrbanFusion: Stochastic Multimodal Fusion for Contrastive Learning of Robust Spatial Representations

arXiv:2510.13774v2 Announce Type: replace-cross Abstract: Forecasting urban phenomena such as housing prices and public health indicators requires the effective integration of various geospatial data.

researcharxiv-cs-cv

Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning

arXiv:2606.00105v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress on vision-language tasks, but they may also memorize and expose sensitive o

BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs

arXiv:2605.30900v1 Announce Type: new Abstract: Current multimodal models handle static image recognition well, but intuitive physical reasoning remains a weakness. Predicting how objects will move an

ConSensus: Multi-Agent Collaboration for Multimodal Sensing

arXiv:2601.06453v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly grounded in sensor data to perceive and reason about human physiology and the physical world. However,

PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selection

arXiv:2502.12119v4 Announce Type: replace-cross Abstract: Visual instruction tuning adapts pre-trained Multimodal Large Language Models (MLLMs) to follow human instructions for real-world applications

applicationsarxiv-cs-ai

TRINE: A Token-Aware, Runtime-Adaptive FPGA Inference Engine for Multimodal AI

arXiv:2603.22867v1 Announce Type: cross Abstract: Multimodal stacks that mix ViTs, CNNs, GNNs, and transformer NLP strain embedded platforms because their compute/memory patterns diverge and hard real

Empathic Prompting: Non-Verbal Context Integration for Multimodal LLM Conversations

arXiv:2510.20743v2 Announce Type: replace-cross Abstract: We present Empathic Prompting, a novel framework for multimodal human-AI interaction that enriches Large Language Model (LLM) conversations wi

MIRAGE: Adaptive Multimodal Gating for Whole-Brain fMRI Encoding

arXiv:2605.29850v1 Announce Type: new Abstract: Recent progress in task-optimized neural networks has established encoding models as a powerful tool for predicting brain responses to naturalistic stim

researcharxiv-cs-lg

TANDEM: Temporal-Aware Neural Detection for Multimodal Hate Speech

arXiv:2601.11178v2 Announce Type: replace Abstract: Social media platforms are increasingly dominated by long-form multimodal content, where harmful narratives are constructed through a complex interp

TRACER: Persistent Regularization for Robust Multimodal Finetuning

arXiv:2605.29380v1 Announce Type: cross Abstract: Mainstream strategies for finetuning pretrained multimodal models often degrade out-of-distribution (OOD) robustness, a phenomenon known as catastroph

EgoBench: An Interactive Egocentric Multimodal Benchmark for Tool-Using Agents

arXiv:2605.27820v1 Announce Type: new Abstract: As AI agents increasingly operate in open, real-world environments, they require a deep synergy of multimodal perception, tool invocation with multi-hop

Information-theoretic Multimodal Representation Learning for Electrocardiogram Signals

arXiv:2605.27583v1 Announce Type: new Abstract: Electrocardiograms (ECGs) are widely used non-invasive measurements of cardiac activity and play a central role in clinical diagnosis. Recent multimodal

safetyarxiv-cs-lg

Local Ai

MMRad-22K: A Structured Multimodal Evidence Dataset for Chest X-ray Report Generation

arXiv:2602.12843v2 Announce Type: replace Abstract: Chest X-ray (CXR) reporting follows a region-based clinical workflow in which radiologists inspect anatomical regions and integrate localized findin

local-aiarxiv-cs-cv

Reasoning Matters: Mitigate Hallucination in Multimodal Large Reasoning Models via Reasoning-Conditioned Preference Optimization

arXiv:2605.27906v1 Announce Type: new Abstract: Multimodal Large Reasoning Models introduce the reasoning paradigm, demonstrating strong capabilities on complex vision-language tasks. However, they st

SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning

arXiv:2602.01990v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) achieve strong performance through instruction tuning, but real-world deployment requires them to con

SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding

arXiv:2601.21666v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) are a major focus of recent AI research. However, most prior work focuses on static image understanding, wh

Utility-Aware Multimodal Contrastive Learning for Product Image Generation

arXiv:2605.28733v1 Announce Type: new Abstract: Product images strongly influence consumer decision-making in online marketplaces. Empowered by multimodal contrastive learning, generative AI can outpu

ViCA: Efficient Multimodal LLMs with Vision-Only Cross-Attention

arXiv:2602.07574v2 Announce Type: replace-cross Abstract: Modern multimodal large language models (MLLMs) adopt a unified self-attention design that processes visual and textual tokens at every Transf

researcharxiv-cs-cl

Agents

Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactions

arXiv:2605.26256v1 Announce Type: new Abstract: Multimodal large language model (MLLM)-based embodied agents have shown strong potential for solving complex tasks in physical environments. However, pe

agentsarxiv-cs-ai

27 May 2026

Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks

arXiv:2505.24876v2 Announce Type: replace-cross Abstract: Deep reasoning is fundamental for solving complex tasks, especially in vision-centric scenarios that demand sequential, multimodal understandi

AnatomiX, an Anatomy-Aware Grounded Multimodal Large Language Model for Chest X-Ray Interpretation

arXiv:2601.03191v3 Announce Type: replace-cross Abstract: Multimodal medical large language models have shown substantial progress in chest X-ray interpretation but continue to face challenges in spat

Hardware

Beyond the Aggregation Dilemma: Prior-Retaining Decoupled Learning for Multimodal Graphs

arXiv:2605.24684v1 Announce Type: cross Abstract: Multimodal Attributed Graph Learning (MAGL) integrates intrinsic node attributes with structural topology via graph aggregation. However, as pretraine

hardwarearxiv-cs-ai

BigMac: Breaking the Pareto Frontier of Compute and Memory in Multimodal LLM Training

arXiv:2605.25451v1 Announce Type: new Abstract: Training multimodal large language models (MLLMs) is challenged by both model and data heterogeneity. Existing systems redesign the training pipeline to

researcharxiv-cs-lg

ConceptM^3oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology

arXiv:2605.24399v1 Announce Type: new Abstract: Healthcare models are transitioning from unimodal prediction toward multimodal reasoning over heterogeneous diagnostic inputs. In computational patholog

applicationsarxiv-cs-ai

Multimodal Functional Maximum Correlation for Emotion Recognition

arXiv:2512.23076v2 Announce Type: replace-cross Abstract: Emotional states manifest as coordinated yet heterogeneous physiological responses across central and autonomic systems, posing a fundamental