Search: “multimodal”

How Far Are We from Generating Missing Modalities with Foundation Models?

model-releasesarxiv-cs-cl

arXiv:2506.03530v3 Announce Type: replace-cross Abstract: Multimodal foundation models have demonstrated impressive capabilities across diverse tasks. However, their potential as plug-and-play solutio

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Mitigating Object Hallucinations via Sentence-Level Early Intervention

DGX agent

arXiv:2507.12455v3 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have revolutionized cross-modal understanding but continue to struggle with hallucinations - fabricated con

researcharxiv-cs-cv

25 May 2026

PGT: Procedurally Generated Tasks for improving visual grounding in MLLMs

model-releasesarxiv-cs-ai

arXiv:2605.23883v1 Announce Type: cross Abstract: Despite remarkable progress in Multimodal Large Language Models (MLLMs), these models still struggle with fine-grained understanding tasks. In this wo

RS2AD-LiDAR: End-to-End Autonomous Driving LiDAR Data Generation from Roadside Sensor Observations

arXiv:2605.23406v1 Announce Type: new Abstract: End-to-end autonomous driving solutions, which directly process multimodal sensory data and output fine-grained control commands, have gradually become

agentsarxiv-cs-cv

TCAP: Tri-Component Attention Profiling for Unsupervised Backdoor Detection in MLLM Fine-Tuning

arXiv:2601.21692v2 Announce Type: replace Abstract: Fine-Tuning-as-a-Service (FTaaS) facilitates the customization of Multimodal Large Language Models (MLLMs) but introduces critical backdoor risks vi

researcharxiv-cs-ai

VGAS: Value-Guided Action-Chunk Selection for Few-Shot Vision-Language-Action Adaptation

arXiv:2602.07399v2 Announce Type: replace Abstract: Vision--Language--Action (VLA) models bridge multimodal reasoning with physical control, but adapting them to new tasks with scarce demonstrations r

researcharxiv-cs-ai

Frequency-Domain Regularized Adversarial Alignment for Transferable Attacks against Closed-Source MLLMs

model-releasesarxiv-cs-lg

arXiv:2605.21541v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) remain vulnerable to transfer-based targeted attacks, where perturbations optimized on open-source surrogate

23 May 2026

Tutorials

Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models

arXiv:2605.22679v1 Announce Type: new Abstract: Vision-language models learn powerful multimodal embeddings, yet their internal semantics remain opaque. While sparse autoencoders (SAEs) can extract in

tutorialsarxiv-cs-cv

Diverse Yet Consistent: Context-Guided Diffusion with Energy-Based Joint Refinement for Multi-Agent Motion Prediction

arXiv:2605.22017v1 Announce Type: new Abstract: Deepgenerative models havebecomeapromisingapproach for human motion prediction due to their ability to capture multimodal distributions and represent di

Enhancing Visual Token Representations for Video Large Language Models via Training-Free Spatial-Temporal Pooling and Gridding

arXiv:2605.22078v1 Announce Type: cross Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have significantly advanced video understanding tasks, yet challenges remain in efficientl

EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning

arXiv:2605.22208v1 Announce Type: new Abstract: Multimodal Large Language Model (MLLM)-driven image restoration agent demonstrates effectiveness in degradation coupling scenarios by flexibly selecting

agentsarxiv-cs-cv

Exposing Vulnerabilities in Visible-Infrared VLMs: A Unified Geometric Adversarial Framework with Cross-Task Transferability

arXiv:2605.22273v1 Announce Type: new Abstract: Vision-language models (VLMs) have achieved strong performance across diverse multimodal tasks, but their adversarial robustness in visible-infrared (VI

applicationsarxiv-cs-cv

LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning

arXiv:2605.22012v1 Announce Type: new Abstract: Joint audio-visual reasoning is essential for omnimodal understanding, yet current multimodal large language models (MLLMs) still struggle when reasonin

researcharxiv-cs-cl

Maestro: Reinforcement Learning to Orchestrate Hierarchical Model-Skill Ensembles

model-releasesarxiv-cs-cl

arXiv:2605.22177v1 Announce Type: cross Abstract: The proliferation of large language models (LLMs) and modular skills has endowed autonomous agents with increasingly powerful capabilities. Existing f

MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues

arXiv:2605.21954v1 Announce Type: new Abstract: Video temporal grounding (VTG), which localizes the start and end times of a queried event in an untrimmed video, is a key test of whether multimodal la

Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality?

arXiv:2605.22109v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in human-facing roles where personality perception is critical, yet existing benchm

SpaceDG: Benchmarking Spatial Intelligence under Visual Degradation

model-releasesarxiv-cs-cl

arXiv:2605.22536v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have made rapid progress in spatial intelligence, yet existing spatial reasoning benchmarks largely assume pr

ST-SimDiff: Balancing Spatiotemporal Similarity and Difference for Efficient Video Understanding with MLLMs

arXiv:2605.22158v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) face significant computational overhead when processing long videos due to the massive number of visual token

Tutorials

VChain: Chain-of-Visual-Thought for Reasoning in Video Generation

arXiv:2510.05094v2 Announce Type: replace Abstract: Recent video generation models can produce smooth and visually appealing clips, but they often struggle to synthesize complex dynamics with a cohere

tutorialsarxiv-cs-cv

VGenST-Bench: A Benchmark for Spatio-Temporal Reasoning via Active Video Synthesis

arXiv:2605.22570v1 Announce Type: new Abstract: Spatio-temporal reasoning is a core capability for Multimodal Large Language Models (MLLMs) operating in the real world. As such, evaluating it precisel

Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoning

arXiv:2601.23224v2 Announce Type: replace Abstract: Existing multimodal large language models for long-video understanding predominantly rely on uniform sampling and single-turn inference, limiting th

VisPhyWorld: Probing Physical Reasoning via Code-Driven Video Reconstruction

arXiv:2602.13294v3 Announce Type: replace Abstract: Evaluating whether Multimodal Large Language Models (MLLMs) genuinely reason about physical dynamics remains challenging. Most existing benchmarks r

Agentic Physical AI toward a Domain-Specific Foundation Model for Nuclear Reactor Control

model-releasesarxiv-cs-lg

arXiv:2512.23292v3 Announce Type: replace-cross Abstract: The prevailing paradigm in AI for physical systems (scaling general-purpose foundation models toward universal multimodal reasoning) confronts

IndusAgent: Reinforcing Open-Vocabulary Industrial Anomaly Detection with Agentic Tools

arXiv:2605.20682v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have shown remarkable capability in bridging visual perception and textual reasoning, enabling zero-shot unders

local-aiarxiv-cs-cv

LTX 2.3 + LTX Director is a Huge improvement

local-air-stablediffusion

LTX-2.3 is a multimodal video generation model developed by Lightricks that generates synchronized audio and video in a single forward pass at resolutions up to 4K at 50 frames per second. The model i

Miller-Index-Based Latent Crystallographic Fracture Plane Reasoning with Vision-Language Models

arXiv:2605.20416v1 Announce Type: new Abstract: We study whether multimodal large language models (MLLMs) can leverage crystallographic plane indices (Miller indices) as a structured latent representa

applicationsarxiv-cs-lg

Modality-Decoupled Online Recursive Editing

arXiv:2605.20273v1 Announce Type: new Abstract: Online model editing for multimodal large language models (MLLMs) requires assimilating a stream of corrections under tight compute and memory budgets.

local-aiarxiv-cs-lg

Ordering Matters: Rank-Aware Selective Fusion for Blended Emotion Recognition

arXiv:2605.21417v1 Announce Type: new Abstract: Blended emotion recognition is challenging because emotions are often expressed as mixtures of subtle and overlapping multimodal cues rather than a sing

ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning

arXiv:2605.20342v1 Announce Type: new Abstract: Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools (e.g., cropping) has become a promisin

agentsarxiv-cs-cv

Safety

Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysis

arXiv:2605.20277v1 Announce Type: new Abstract: Medical vision-language models (VLMs) have rapidly advanced as general-purpose multimodal assistants, yet their deployment in 3D Computed Tomography (CT

safetyarxiv-cs-cv

Uni-Edit: Intelligent Editing Is A General Task For Unified Model Tuning

arXiv:2605.21487v1 Announce Type: new Abstract: Currently, enhancing Unified Multimodal Models (UMMs) with image understanding, generation, and editing capabilities mainly relies on mixed multi-task t

VersusQ: Pairwise Margin Reasoning for Generalizable Video Quality Assessment

arXiv:2605.21130v1 Announce Type: new Abstract: Large Multimodal Models (LMMs) have shown promise for video quality assessment, but most methods still predict an absolute score for each video. Such po

VIHD: Visual Intervention-based Hallucination Detection for Medical Visual Question Answering

arXiv:2605.20772v1 Announce Type: new Abstract: While medical Multimodal Large Language Models (MLLMs) have shown promise in assisting diagnosis, they still frequently generate hallucinated responses

AQuaUI: Visual Token Reduction for GUI Agents with Adaptive Quadtrees

arXiv:2605.19260v1 Announce Type: new Abstract: Large Multimodal Models (LMMs) have recently emerged as promising backbones for GUI-agent models, where high-resolution GUI screenshots are introduced t

agentsarxiv-cs-ai

Benchmarking and Evolving Reason-Reflect-Rectify for Reflective Visual Generation

arXiv:2605.19639v1 Announce Type: new Abstract: Text-to-Image (T2I) models and Unified Multimodal Models (UMMs) have achieved remarkable progress in visual generation. However, their reliance on a sin

Causal Evidence for Attention Head Imbalance in Modality Conflict Hallucination

model-releasesarxiv-cs-ai

arXiv:2605.19250v1 Announce Type: new Abstract: Modality-conflict hallucination occurs when multimodal large language models (MLLMs) prioritize erroneous textual premises over contradictory visual evi

CosFly: Plan in the Matrix, Fly in the World

arXiv:2605.19120v1 Announce Type: new Abstract: We present CosFly, a box-structured planning and multimodal simulation pipeline for aerial tracking, together with CosFly-Track, a large-scale UAV datas

researcharxiv-cs-ro

DarkLLM: Learning Language-Driven Adversarial Attacks with Large Language Models

arXiv:2605.18868v1 Announce Type: cross Abstract: While vision and multimodal foundation models underpin critical tasks from perception to complex reasoning, they remain highly vulnerable to adversari

applicationsarxiv-cs-ai

Safety

DynaTok: Temporally Adaptive and Positional Bias-Aware Token Compression for Video-LLMs

arXiv:2605.19322v1 Announce Type: new Abstract: Recent advances in Video Large Language Models (Video-LLMs) have greatly expanded multimodal reasoning capabilities. However, the massive number of visu

safetyarxiv-cs-cv

ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG Interpretation

arXiv:2602.04279v2 Announce Type: replace Abstract: Electrocardiography (ECG) serves as an indispensable diagnostic tool in clinical practice, yet existing multimodal large language models (MLLMs) rem

researcharxiv-cs-cl

Eyes on VLM: Benchmarking Gaze Following and Social Gaze Prediction in Vision Language Models

arXiv:2605.19859v1 Announce Type: new Abstract: Vision-language models (VLMs) have rapidly evolved into general-purpose multimodal reasoners with strong zero-shot generalization. In this context, VLMs

Safety

HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling

arXiv:2510.00054v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have made significant strides in visual understanding tasks. However, their performance on high-resol

safetyarxiv-cs-ai

LMM-Track4D: Eliciting 4D Dynamic Reasoning in LMMs via Trajectory-Grounded Dialogue

arXiv:2605.19390v1 Announce Type: new Abstract: Recent large multimodal models (LMMs) have become increasingly capable on image and video understanding, yet still struggle to sustain 4D continuous spa

Query-Conditioned Graph Retrieval for Contextualized LLM Reasoning in Personalized Wearable Data

arXiv:2605.18763v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly applied to analyzing wearable sensing data, which are long-term, multimodal, and highly personalized. A

local-aiarxiv-cs-ai

Semantic-Enriched Latent Visual Reasoning

arXiv:2605.19342v1 Announce Type: new Abstract: Multimodal latent-space reasoning aims to replace explicit thinking with images by performing visual reasoning directly in a compact latent space. Howev

Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence

arXiv:2505.23747v2 Announce Type: replace-cross Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have significantly enhanced performance on 2D visual tasks. However, improving

applicationsarxiv-cs-ai

Token by Token, Compromised: Backdoor Vulnerabilities in Unified Autoregressive Models

arXiv:2605.19227v1 Announce Type: cross Abstract: Unified autoregressive models (UAMs) are transformer models that generate text as well as image tokens within a single autoregressive pass. Shared par

researcharxiv-cs-ai

Towards Camera-Robust 3D Localization: Equation-Anchored Tool-Use for MLLMs

arXiv:2605.19528v1 Announce Type: new Abstract: 3D localization in Multimodal Large Language Models (MLLMs), including 3D object detection and 3D visual grounding, is fundamentally limited by camera i