Search: “multimodal”

EGRA:Toward Enhanced Behavior Graphs and Representation Alignment for Multimodal Recommendation

arXiv:2508.16170v2 Announce Type: replace-cross Abstract: MultiModal Recommendation (MMR) systems have emerged as a promising solution for improving recommendation quality by leveraging rich item-side

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

GlaBoost: A Multimodal Structured Framework for Glaucoma Risk Stratification

DGX agent

arXiv:2508.03750v2 Announce Type: replace-cross Abstract: Early and accurate glaucoma detection is critical to prevent irreversible vision loss, yet existing AI methods often rely on unimodal inputs a

applicationsarxiv-cs-cv

7 Jul 2026

SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning

model-releasesarxiv-cs-cl

arXiv:2603.23483v2 Announce Type: replace-cross Abstract: Agentic multimodal large language models (MLLMs) (e.g., OpenAI o3 and Gemini Agentic Vision) achieve remarkable reasoning capabilities through

TexTailor: Inference-Time Textual Guidance Tailoring for Multimodal Diffusion Transformers

arXiv:2601.02211v2 Announce Type: replace Abstract: Recent breakthroughs of transformer-based diffusion models, particularly with Multimodal Diffusion Transformers (MMDiT) driven models like FLUX and

The P^3 Dataset: Pixels, Points and Polygons for Multimodal Building Vectorization

arXiv:2505.15379v2 Announce Type: replace Abstract: We present the P^3 dataset, a large-scale multimodal benchmark for building vectorization, constructed from aerial LiDAR point clouds, high-resoluti

Transferability Between Understanding and Generation in Unified Multimodal Models

arXiv:2607.04423v1 Announce Type: cross Abstract: Unified Multimodal Models (UMMs) integrate image understanding and generation within a single architecture, yet how the two tasks interact remains und

researcharxiv-cs-ai

VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs

arXiv:2511.20272v2 Announce Type: replace Abstract: While Multimodal Large Language Models (MLLMs) have become adept at recognizing objects, they often lack the intuitive, human-like understanding of

Agents

Web-CogReasoner: Towards Multimodal Knowledge-Induced Cognitive Reasoning for Web Agents

arXiv:2508.01858v3 Announce Type: replace-cross Abstract: Multimodal large-scale models have significantly advanced the development of web agents, enabling perception and interaction with digital envi

agentsarxiv-cs-ai

WorldBagel: Uncovering the Power of Unified Multimodal Models for Vision-Language-Action-World Modeling

arXiv:2607.03461v1 Announce Type: new Abstract: World models aim to capture environment dynamics in ways that support perception, reasoning, and action, and have recently become a central direction in

Tutorials

Learn why multimodal prompting is a big deal when working with coding agents.

Multimodal prompting enhances coding agents by enabling them to process and integrate multiple types of input data—such as text, images, and code snippets—simultaneously, improving their ability to un

tutorialsdair-ai--x

4 Jul 2026

MMIR-TCM: Memory-Integrated Multimodal Inference and Retrieval for TCM Clinical Decision Support

arXiv:2607.01814v1 Announce Type: new Abstract: Traditional Chinese Medicine (TCM) diagnosis, particularly through tongue inspection, faces persistent challenges in subjectivity and reproducibility. T

3 Jul 2026

Decompose, Compare, and Decide: Multimodal LLMs are Implicit Few-Shot Learners

arXiv:2607.00125v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable abilities when analyzing images, yet translating these capabilities to few-shot im

researcharxiv-cs-cv

2 Jul 2026

Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference

arXiv:2606.31903v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) increasingly process long visual-token sequences, increasing the overall inference computation. Existing acce

researcharxiv-cs-ai

Multimodal Benchmark for Safety Assessment in Industrial Inspection Scenarios

arXiv:2601.21173v2 Announce Type: replace-cross Abstract: With the rapid development of industrial intelligence and unmanned inspection, reliable perception and safety assessment for AI systems in com

On Optimizing Multimodal Jailbreaks for Spoken Language Models

arXiv:2603.19127v2 Announce Type: replace Abstract: As Spoken Language Models (SLMs) integrate speech and text modalities, they inherit the safety vulnerabilities of their LLM backbone while introduci

safetyarxiv-cs-lg

SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Search

arXiv:2606.31504v1 Announce Type: new Abstract: We present SimpleSearch-VL, an efficient, reliable, and practical framework for multimodal agentic search. Its core idea is to improve the agent's own s

UniTac: A Unified Multimodal Model for Cross-Sensor Tactile Understanding and Generation

arXiv:2606.31451v1 Announce Type: cross Abstract: Unified multimodal models (UMMs) have shown great promise in integrating understanding and generation across diverse modalities. However, existing res

Can AI Draw Science? A Benchmark for Evaluating Scientific Figure Generation by Text-to-Image and Multimodal Models

arXiv:2606.28406v1 Announce Type: cross Abstract: Text-to-image and multimodal generative models are increasingly used to produce scientific figures such as mechanism diagrams, experimental-design sch

Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation

arXiv:2606.29805v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) are prone to hallucination as their generation preferences are insufficiently calibrated to visual evidence, ca

GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes

arXiv:2606.28514v1 Announce Type: new Abstract: Multimodal models are increasingly deployed to solve tasks collaboratively with humans or other artificial agents. Existing benchmarks show that these m

MemLeak: Diagnosing Information Leaks in Multimodal Agent Memory

model-releasesarxiv-cs-lg

arXiv:2606.29788v1 Announce Type: new Abstract: When a multimodal AI agent is asked to forget a fact, current memory systems usually delete the text entry and report success. We find that the fact can

Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancement

arXiv:2403.06728v2 Announce Type: replace Abstract: Radiology report generation (RRG) has attracted significant attention due to its potential to reduce the workload of radiologists. The performance o

Reliability-Prioritized Fine-Grained Generation in Multimodal Large

arXiv:2606.29573v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly expected to generate fine-grained descriptions of visual content. However, we observe and theo

SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models

arXiv:2510.12784v2 Announce Type: replace-cross Abstract: Recently, remarkable progress has been made in Unified Multimodal Models (UMMs), which integrate vision-language generation and understanding

safetyarxiv-cs-cl

X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretraining

arXiv:2606.14752v2 Announce Type: replace-cross Abstract: Modern Vision-Language-Action (VLA) models must bridge pretrained vision-language reasoning and precise continuous robot control. Existing act

AirGroundBench: Probing Spatial Intelligence in Multimodal Large Models under Heterogeneous Multi-View Embodied Collaboration

arXiv:2606.28049v1 Announce Type: new Abstract: In recent years, multimodal large language models (MLLMs) have shown strong potential for embodied intelligence, yet their ability to maintain geometric

Lost at the End: Primacy Bias in Multimodal Retrieval-Augmented Question Answering

arXiv:2606.16494v2 Announce Type: replace-cross Abstract: Knowledge-based visual question answering (KB-VQA) lets vision-language systems answer questions that exceed their parametric knowledge by con

MER-R1: Multimodal Emotion Reasoning via Slow-Fast Thinking Synergy

arXiv:2606.27652v1 Announce Type: new Abstract: We find that explicit reasoning does not necessarily translate into better multimodal emotion recognition (MER) accuracy, even though it makes predictio

researcharxiv-cs-ai

Spectral Text Fusion: A Frequency-Aware Approach to Multimodal Time-Series Forecasting

arXiv:2602.01588v3 Announce Type: replace-cross Abstract: Multimodal time series forecasting is crucial in real-world applications, where decisions depend on both numerical data and contextual signals

Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning

arXiv:2511.20196v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) can inadvertently memorize privacy-sensitive information during training. While existing unlearning methods

Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards

arXiv:2606.27376v1 Announce Type: new Abstract: Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision,

Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models

arXiv:2606.27373v1 Announce Type: new Abstract: Recently, self-evolving large multimodal models (LMMs) have received attention for improving visual reasoning in a purely unsupervised setting. However,

Reconstruction Alignment Improves Unified Multimodal Models

arXiv:2509.07295v4 Announce Type: replace-cross Abstract: Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture. However, conventional training relies

SocialPersona: Benchmarking Personalized Profiling and Response with Multimodal Social-Media Context

model-releasesarxiv-cs-cl

arXiv:2606.26654v1 Announce Type: new Abstract: Personalized language-model assistants are often evaluated through a memory lens: can a model recall preferences users have explicitly stated in dialogu

Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning

arXiv:2606.18974v2 Announce Type: replace Abstract: Unified multimodal models (UMMs) interleave generated ''visual thoughts'' (VTs) with text reasoning to improve spatial tasks. This incurs roughly an

We stress tested many frontier AI models for multimodal medical reasoning (including GPT-5, Claude 3.5, Gemini 2.5 Pro). They’re not ready. …

model-releasesgary-marcus--x

We stress tested many frontier AI models for multimodal medical reasoning (including GPT-5, Claude 3.5, Gemini 2.5 Pro). They’re not ready. Faulty reasoning, use of inappropriate shortcuts, hallucinat

When Does Quality-Aware Multimodal Fusion Matter? A Leakage-Safe Diagnostic for Decision-Level Dependence

arXiv:2606.26473v1 Announce Type: new Abstract: Many multimodal systems estimate the reliability of each modality and weight their contributions to the final prediction. However, it remains unclear wh

researcharxiv-cs-lg

Beyond Visual Forensics: Auditing Multimodal Robustness for Synthetic Medical Image Detection

arXiv:2606.25375v1 Announce Type: new Abstract: With the rapid adoption of generative AI, synthetic medical images pose growing risks, including diagnostic deception and insurance fraud. Although prio

25 Jun 2026

Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Models

model-releasesarxiv-cs-cl

arXiv:2606.26079v1 Announce Type: new Abstract: Standard benchmarks for multimodal large language models (MLLMs) score each item on one canonical ordering and miss whether order-irrelevant shuffling c

25 Jun 2026

Latent Visual States for Efficient Multimodal Reasoning

arXiv:2606.24233v1 Announce Type: new Abstract: The integration of visual evidence has significantly enhanced the capabilities of large multimodal models. However, this integration predominantly relie

24 Jun 2026

Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Sampling

arXiv:2606.24187v1 Announce Type: new Abstract: Long video understanding remains a daunting challenge for Multimodal Large Language Models (MLLMs) due to the excessive computation and memory footprint

24 Jun 2026

Catching Lies Without Sending the Video: Privacy-Preserving Multimodal Deception Detection

arXiv:2606.22699v1 Announce Type: new Abstract: Frontier multimodal models can guess whether a person is lying from a testimony video. To do so, they stream that raw face and voice to a third-party mo

CFPO: Counterfactual Policy Optimization for Multimodal Reasoning

arXiv:2606.23206v1 Announce Type: new Abstract: Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in multimodal reasoning. However, prevailing reinforcement learning (RL)

DamageArbiter: A Multimodal Arbitration Framework for Disaster Damage Assessment from Street-View Imagery

arXiv:2603.14837v2 Announce Type: replace Abstract: Analyzing street-view imagery with computer vision models offers a promising approach for rapid, hyperlocal disaster damage assessment, but existing

researcharxiv-cs-cv

DataClaw0: Agentic Tailoring Multimodal Data from Raw Streams

model-releasesarxiv-cs-lg

arXiv:2606.21337v1 Announce Type: new Abstract: Massive unstructured multimodal streams suffer from high 'data entropy,' impeding both efficient human knowledge acquisition and high-quality AI post-tr

Real-Time Multimodal Activity-Aware Error Detection in Robot-Assisted Surgery

arXiv:2606.23593v1 Announce Type: cross Abstract: Robot-assisted minimally invasive surgery improves surgical precision but introduces complexity, making technical error detection essential for ensuri

From 2D Grids to 1D Tokens: Reforming Shared Representations for Multimodal Image Fusion

arXiv:2606.12303v1 Announce Type: new Abstract: Multimodal image fusion aims to integrate complementary information from different modalities into a fused image that preserves rich local details while

researcharxiv-cs-cv

11 Jun 2026

5% > 100%: Flatness Preference is All You Need for Multimodal Parameter-Efficient Fine-Tuning

arXiv:2606.10488v1 Announce Type: new Abstract: Parameter-Efficient Fine-Tuning (PEFT) methods provide a streamlined and efficient tool for adapting large models to domain-specific multimodal downstre