Search: “multimodal”

DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories

arXiv:2602.10809v2 Announce Type: replace Abstract: Existing multimodal retrieval systems excel at semantic matching but implicitly assume that query-image relevance can be measured in isolation. This

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agents

Demo2: Multimodal Interactive Hybrid Agent

DGX agent

Demo2 showcases Qwen's multimodal interactive hybrid agent capabilities, likely demonstrating the integration of multiple data types (text, image, audio, video) with interactive features and hybrid pr

agentsqwen--x

1 Jun 2026

ERGeoBench:A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models

arXiv:2605.31251v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have shown strong potential as embodied agents, yet embodied geo-localization remains underexplored due to th

In May, we integrated 11 new models spanning image, 3D, audio, video, and multimodal. The highlights: → Krea 2 — style-first image generatio…

In May, we integrated 11 new models spanning image, 3D, audio, video, and multimodal. The highlights: → Krea 2 — style-first image generation, live as a Partner Node on day one. Competes on how the fr

model-releasescomfyui--x

Learning from Fine-Grained Visual Discrepancies: Mitigating Multimodal Hallucinations via In-Context Visual Contrastive Optimization

arXiv:2605.31312v1 Announce Type: cross Abstract: Multimodal hallucination remains a persistent challenge for Vision-Language Models (VLMs). Standard textual Direct Preference Optimization (DPO) often

researcharxiv-cs-cl

Applications

MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding

arXiv:2605.30794v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated significant achievements in general visual question answering (VQA) tasks. However, they re

applicationsarxiv-cs-ai

Multimodal Fusion via Self-Consistent Task-Gradient Fields

arXiv:2410.15475v2 Announce Type: replace Abstract: Multimodal learning aims to preserve as much task-related information as possible from different inputs. However, current fusion designs often disto

researcharxiv-cs-cv

Local Ai

Reasoning-Aware Multimodal Fusion for Hateful Video Detection

arXiv:2512.02743v2 Announce Type: replace-cross Abstract: Hate speech in online videos is posing an increasingly serious threat to digital platforms, especially as video content becomes increasingly m

local-aiarxiv-cs-ai

Tutorials

Representation Forcing for Bottleneck-Free Unified Multimodal Models

arXiv:2605.31604v1 Announce Type: new Abstract: Unified multimodal models (UMMs) aim to handle perception and generation in a single model. Yet existing UMMs still rely on a frozen, separately pretrai

tutorialsarxiv-cs-cv

Rethinking Multimodal Few-Shot 3D Point Cloud Segmentation: From Fused Refinement to Decoupled Arbitration

arXiv:2601.01456v2 Announce Type: replace-cross Abstract: In this paper, we revisit multimodal few-shot 3D point cloud semantic segmentation (FS-PCS), identifying a conflict in 'Fuse-then-Refine' para

Applications

VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoning

arXiv:2605.31457v1 Announce Type: new Abstract: With the rapid advancement of large multimodal models (LMMs), inference-time overhead has become a key bottleneck for real-world deployment. Existing me

applicationsarxiv-cs-cv

Analyzing Persona Effects in Generated Explanations from Multimodal LLM Agents in Urban Perception

arXiv:2605.29064v1 Announce Type: new Abstract: We study how persona prompting shapes language generated by multimodal large language models in an urban perception setting. Using 59,808 annotations fr

researcharxiv-cs-cl

Benchmarking Large Vision-Language Models on CFMME: A Comprehensive Chinese Financial Multimodal Evaluation Dataset

arXiv:2605.29462v1 Announce Type: cross Abstract: The emergence of Large Vision-Language Models (LVLMs) has substantially expanded model capabilities beyond text-only understanding, enabling unified i

DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmark

arXiv:2605.30027v1 Announce Type: new Abstract: Multimodal documents contain diverse elements, such as tables, figures, and layouts, which can complicate retrieval tasks. While current approaches typi

HiKEY: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answering

arXiv:2605.29606v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) for document-based Open-domain Question Answering (ODQA) on large-scale industrial corpora faces two critical bottl

MuPHI: Learning Implicit Multimodal Harm Reasoning via Semantically Grounded Reward Optimization

arXiv:2605.29951v1 Announce Type: new Abstract: Understanding how harm emerges from interaction between otherwise benign image-text pairs requires intent-aware cross-modal reasoning beyond surface-lev

Agents

Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation

arXiv:2605.29861v1 Announce Type: cross Abstract: Large Language Models (LLMs) have advanced autonomous agents from deep search, which retrieves concise factual answers, to deep research, which synthe

agentsarxiv-cs-ai

A Conflict-Aware Penalty and Statistical Loss Framework for Balancing Modalities and Enhancing Stability in Multimodal Sentiment Analysis

arXiv:2605.28575v1 Announce Type: new Abstract: Multimodal Sentiment Analysis (MSA) fuses text, acoustic, and visual streams to infer sentiment. Because pre-trained text encoders are far more expressi

DREAM-R: Multimodal Speculative Reasoning with RL-Based Refined Drafting, Precise Verification, and Fully Parallel Execution

arXiv:2605.28678v1 Announce Type: new Abstract: Speculative reasoning has recently been proposed as a means to accelerate reasoning-intensive generation in large multimodal models, but its effectivene

EVADE-Bench: Multimodal Benchmark for Evaluating and Enhancing Evasive Content Detection

arXiv:2505.17654v4 Announce Type: replace-cross Abstract: E-commerce platforms increasingly rely on Large Language Models (LLMs) and Vision Language Models (VLMs) to detect illicit or misleading produ

Mags-RL: Wearing Multimodal LLMs a Magnifying Glass via Agentic Reinforcement Learning For Complex Scene Reasoning

arXiv:2605.27960v1 Announce Type: new Abstract: Despite their popularity and success, Multimodal Large Language Models (MLLMs) often struggle to interpret images accurately, which limits their reasoni

RE-TRIANGLE: Does TRIANGLE Enable Multimodal Alignment Beyond Cosine Similarity in Retrieval?

arXiv:2605.27436v1 Announce Type: cross Abstract: Multimodal alignment is critical for bridging the semantic gap in information retrieval. However, traditional pairwise strategies introduce a geometri

REVEAL: Reference-Grounded Reasoning for Multimodal Manipulation Detection

arXiv:2605.28459v1 Announce Type: new Abstract: Multimodal manipulation detection aims to simultaneously identify forged image--text pairs and localize tampered regions, yet existing methods typically

researcharxiv-cs-cv

Advancing Creative Physical Intelligence in Large Multimodal Models

arXiv:2605.26396v1 Announce Type: new Abstract: Large multimodal models (LMMs) have rapidly advanced in perception and reasoning; however, it remains unclear whether these capabilities generalize to d

Athena: Enhancing Multimodal Reasoning with Data-efficient Process Reward Models

arXiv:2506.09532v5 Announce Type: replace-cross Abstract: We present Athena-PRM, a multimodal process reward model (PRM) designed to evaluate the reward score for each step in solving complex reasonin

Tutorials

DynFrame: Adaptive Reasoning-Driven Multimodal Framework with Dynamic Frame Augmentation for Complex Video Understanding

arXiv:2605.26680v1 Announce Type: cross Abstract: Recent video multimodal large language models (MLLMs) increasingly couple step-by-step reasoning with on-demand visual evidence retrieval, allowing mo

tutorialsarxiv-cs-ai

LiveK12Bench: Have Large Multimodal Models Truly Conquered High School-level Examinations?

arXiv:2605.26781v1 Announce Type: new Abstract: Advanced Large Multimodal Models (LMMs) have demonstrated impressive performance in K-12 reasoning tasks, exhibiting great promise as intelligent tutors

A Multimodal Framework for Dementia Detection via Linguistic and Acoustic Representation Learning

arXiv:2605.25540v1 Announce Type: cross Abstract: Alzheimer's disease (AD) is a progressive neurodegenerative disorder and the leading cause of dementia, affecting memory, reasoning, communication, an

safetyarxiv-cs-lg

Local Ai

Divide-and-Conquer Inference for Large-Scale Visual Recognition with Multimodal Large Language Models

arXiv:2605.24799v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities across a wide range of vision language tasks. However, when applied to

local-aiarxiv-cs-ai

Do Understanding and Generation Fight? A Diagnostic Study of DPO for Unified Multimodal Models

arXiv:2603.17044v2 Announce Type: replace-cross Abstract: Unified multimodal models share a language model backbone for both understanding and generating images. Can DPO align both capabilities simult

Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generation

arXiv:2601.21406v3 Announce Type: replace-cross Abstract: Unified Multimodal Models (UMMs) integrate both visual understanding and generation within a single framework. Their ultimate aspiration is to

researcharxiv-cs-lg

M^3-Verse: A 'Spot the Difference' Challenge for Large Multimodal Models

arXiv:2512.18735v2 Announce Type: replace-cross Abstract: Modern Large Multimodal Models (LMMs) have demonstrated extraordinary ability in static image and single-state spatial-temporal understanding.

Tutorials

Mode-as-Sequence: Translating Multimodal Motion Prediction into Unified Sequential Mode Modeling

arXiv:2605.24037v1 Announce Type: cross Abstract: Multimodal motion forecasting is inherently under-supervised: each training scene provides only one realized future, yet multiple plausible futures ex

tutorialsarxiv-cs-ai

MuCRASP: Multimodal Chain-of-thought Reasoning aware Structured Pruning

arXiv:2605.25842v1 Announce Type: new Abstract: Vision-language models (VLMs) increasingly rely on chain-of-thought (CoT) reasoning to solve complex multimodal tasks, but their large parameter sizes m

Agents

PANDO: Efficient Multimodal AI Agents via Online Skill Distillation

arXiv:2605.24785v1 Announce Type: new Abstract: Recent advances in multimodal web agents often rely on increased inference-time computation, including rollout search, verifier passes, offline skill di

agentsarxiv-cs-ai

Parameter Efficient Multi-Class Intelligent Scheduling for Multimodal Online Distributed Industrial Anomaly Detection

arXiv:2605.23984v1 Announce Type: cross Abstract: Industrial anomaly detection has attracted significant attention as a fundamental challenge in industrial systems. The rapid advancement of heterogene

Squeezing Capacity from Multimodal Large Language Models for Subject-driven Generation

arXiv:2605.26111v1 Announce Type: cross Abstract: Subject-driven image generation aims to synthesize new images that preserve the identity of the given subject while following textual instructions. Ex

CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception

arXiv:2605.23655v1 Announce Type: cross Abstract: High-resolution (HR) image perception presents a key bottleneck for multimodal large language models (MLLMs). While visual search offers a promising s

25 May 2026

Multimodal Crystal Flow: Any-to-Any Modality Generation for Unified Crystal Modeling

arXiv:2602.20210v2 Announce Type: replace-cross Abstract: Crystal modeling spans a family of conditional and unconditional generation tasks, including crystal structure prediction (CSP) and de novo ge

25 May 2026

Open Multimodal Datasets and Open-Source Software for Data-Driven Modeling of Multiphase Transport and Thermal Systems

model-releasesarxiv-cs-lg

arXiv:2605.23037v1 Announce Type: new Abstract: Data-driven modeling is becoming central to multiphase transport, electronics cooling, acoustic diagnostics, and thermal-fluid digital twins, but progre

25 May 2026

Thinking Machines is impressive. In a couple hours I just fine tuned my own Qwen3.5-397B model this afternoon. Fast usable multimodal is als…

researchsoumith-chintala--x

Thinking Machines is impressive. In a couple hours I just fine tuned my own Qwen3.5-397B model this afternoon. Fast usable multimodal is also going to enable very mind-blowing personal AI. People talk

24 May 2026

Distributed Image Compression with Multimodal Side Information at Extremely Low Bitrates

arXiv:2605.22061v1 Announce Type: new Abstract: Distributed Image Compression (DIC) is crucial for multi-view transmission, especially when operating at extremely low bitrates (< 0.1 bpp). Its core ch

safetyarxiv-cs-cv

Enhancing Multimodal Large Language Models for Safety-Critical Driving Video Analysis

arXiv:2605.22185v1 Announce Type: new Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in general visual understanding. However, thei

safetyarxiv-cs-cv

Agents

Evaluating multimodal emotion recognition in proactive conversational agents: A user study

arXiv:2605.20200v1 Announce Type: cross Abstract: This article presents a multimodal emotion recognition module integrated into a proactive Socially Interactive Agent (SIA) powered by generative artif

agentsarxiv-cs-ai

I think people don't realize why Gemini Omni is different than other video AIs. It is fully multimodal, so it can edit video natively, too I…

model-releasesethan-mollick--x

I think people don't realize why Gemini Omni is different than other video AIs. It is fully multimodal, so it can edit video natively, too I took the famous 'train ' movie from 1896 & made it a bullet

MOTOR: A Multimodal Dataset for Two-Wheeler Rider Behavior Understanding

arXiv:2605.22550v1 Announce Type: new Abstract: Two-wheelers account for a disproportionately high share of road fatalities in the Global South. Research on two-wheeler rider behavior, however, lags f

Understanding Multimodal Failure in Action-Chunking Behavioral Cloning

arXiv:2605.22493v1 Announce Type: cross Abstract: Behavioral cloning becomes difficult when the same observation admits several valid actions. We study this problem for action-chunking policies and sh

researcharxiv-cs-ro

Deeper Thought, Weaker Aim: Understanding and Mitigating Perceptual Impairment during Reasoning in Multimodal Large Language Models