Search: “multimodal”

Making Dialogue Grounding Data Rich: A Three-Tier Data Synthesis Framework for Generalized Referring Expression Comprehension

arXiv:2512.02791v2 Announce Type: replace Abstract: Dialogue-Based Generalized Referring Expression Comprehension (GREC) requires models to ground the expression and unlimited targets in complex visua

researcharxiv-cs-cl

28 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

MIRAGE: A Micro-Interaction Relational Architecture for Grounded Exploration in Multi-Figure Artworks

DGX agent

arXiv:2604.23788v1 Announce Type: new Abstract: Appreciating multi-figure paintings requires understanding how characters relate through subtle cues like gaze alignment, gesture, and spatial arrangeme

safetyarxiv-cs-cv

28 Apr 2026

BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answering

arXiv:2604.22678v1 Announce Type: new Abstract: A common approach to question answering with retrieval-augmented generation (RAG) is to concatenate documents into a single context and pass it to a lan

researcharxiv-cs-cl

27 Apr 2026

Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning Segmentation

arXiv:2510.19592v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) demonstrate strong video understanding by attending to visual tokens relevant to textual queries. To direct

27 Apr 2026

ReCAPA: Hierarchical Predictive Correction to Mitigate Cascading Failures

arXiv:2604.21232v1 Announce Type: new Abstract: Vision-Language-Action systems follow instructions to execute multi-step tasks in multimodal environments. Recent VLA approaches typically rely on post-

local-aiarxiv-cs-ai

24 Apr 2026

SGG-R^{rm 3}: From Next-Token Prediction to End-to-End Unbiased Scene Graph Generation

arXiv:2603.07961v3 Announce Type: replace Abstract: Scene Graph Generation (SGG) structures visual scenes as graphs of objects and their relations. While Multimodal Large Language Models (MLLMs) have

24 Apr 2026

Rethinking Reinforcement Fine-Tuning in LVLM: Convergence, Reward Decomposition, and Generalization

arXiv:2604.19857v1 Announce Type: cross Abstract: Reinforcement fine-tuning with verifiable rewards (RLVR) has emerged as a powerful paradigm for equipping large vision-language models (LVLMs) with ag

safetyarxiv-cs-cl

23 Apr 2026

Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Models

arXiv:2601.22737v2 Announce Type: replace Abstract: The robust safety of Vision-Language Large Models (VLLMs) against joint multilingual and multimodal threats remains severely underexplored. Current

22 Apr 2026

BOOKAGENT: Orchestrating Safety-Aware Visual Narratives via Multi-Agent Cognitive Calibration

arXiv:2604.16541v1 Announce Type: new Abstract: Recent advancements in Large Generative Models (LGMs) have revolutionized multi-modal generation. However, generating illustrated storybooks remains an

Emotion Collider: Dual Hyperbolic Mirror Manifolds for Sentiment Recovery via Anti Emotion Reflection

arXiv:2602.16161v3 Announce Type: replace-cross Abstract: Emotional expression underpins natural communication and effective human-computer interaction. We present Emotion Collider (EC-Net), a hyperbo

researcharxiv-cs-cl

End-to-end Listen, Look, Speak and Act

model-releasesarxiv-cs-cl

arXiv:2510.16756v2 Announce Type: replace-cross Abstract: Human interaction is inherently multimodal and full-duplex: we listen while watching, speak while acting, and fluidly adapt to turn-taking and

Geometry-Guided 3D Visual Token Pruning for Video-Language Models

arXiv:2604.18260v1 Announce Type: new Abstract: Multimodal large language models have demonstrated remarkable capabilities in 2D vision, motivating their extension to 3D scene understanding. Recent st

OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models

arXiv:2511.14582v2 Announce Type: replace Abstract: Omnimodal large language models (OmniLLMs) have attracted increasing research attention of late towards unified audio-video understanding. However,

Real-Time Visual Attribution Streaming in Thinking Model

arXiv:2604.16587v1 Announce Type: new Abstract: We present an amortized framework for real-time visual attribution streaming in multimodal thinking models. When these models generate code from a scree

Joint-Centric Dual Contrastive Alignment with Structure-Preserving and Information-Balanced Regularization

arXiv:2604.16247v1 Announce Type: cross Abstract: We propose HILBERT (HIerarchical Long-sequence Balanced Embedding with Reciprocal contrastive Training), a cross-attentive multimodal framework for le

20 Apr 2026

VeRVE: Versatile Retrieval for Videos via Unified Embeddings

arXiv:2601.12193v3 Announce Type: replace Abstract: Modern video retrieval systems are expected to handle diverse tasks ranging from corpus-level retrieval, fine-grained moment localization to flexibl

local-aiarxiv-cs-cv

20 Apr 2026

CaptionQA: Is Your Caption as Useful as the Image Itself?

arXiv:2511.21025v2 Announce Type: replace Abstract: Image captions serve as efficient surrogates for visual content in multimodal systems such as retrieval, recommendation, and multi-step agentic infe

Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models

arXiv:2604.14888v1 Announce Type: new Abstract: Recent advances in vision language models (VLMs) offer reasoning capabilities, yet how these unfold and integrate visual and textual information remains

safetyarxiv-cs-cl

Towards Scalable Lightweight GUI Agents via Multi-role Orchestration

arXiv:2604.13488v1 Announce Type: new Abstract: Autonomous Graphical User Interface (GUI) agents powered by Multimodal Large Language Models (MLLMs) enable digital automation on end-user devices. Whil

V-Reflection: Transforming MLLMs from Passive Observers to Active Interrogators

arXiv:2604.03307v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable success, yet they remain prone to perception-related hallucinations in fine-graine

local-aiarxiv-cs-cv

Character Beyond Speech: Leveraging Role-Playing Evaluation in Audio Large Language Models via Reinforcement Learning

arXiv:2604.13804v1 Announce Type: new Abstract: The rapid evolution of multimodal large models has revolutionized the simulation of diverse characters in speech dialogue systems, enabling a novel inte

safetyarxiv-cs-lg

UMI-3D: Extending Universal Manipulation Interface from Vision-Limited to 3D Spatial Perception

arXiv:2604.14089v1 Announce Type: new Abstract: We present UMI-3D, a multimodal extension of the Universal Manipulation Interface (UMI) for robust and scalable data collection in embodied manipulation

safetyarxiv-cs-ro

ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Body

arXiv:2512.14234v2 Announce Type: replace Abstract: Human communication is inherently multimodal and social: words, prosody, and body language jointly carry intent. Yet most prior systems model human

Visual Self-Fulfilling Alignment: Shaping Safety-Oriented Personas via Threat-Related Images

arXiv:2603.08486v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) face safety misalignment, where visual inputs enable harmful outputs. To address this, existing methods req

Every Picture Tells a Dangerous Story: Memory-Augmented Multi-Agent Jailbreak Attacks on VLMs

arXiv:2604.12616v1 Announce Type: new Abstract: The rapid evolution of Vision-Language Models (VLMs) has catalyzed unprecedented capabilities in artificial intelligence; however, this continuous modal

15 Apr 2026

Bridging the RGB-IR Gap: Consensus and Discrepancy Modeling for Text-Guided Multispectral Detection

arXiv:2604.11234v1 Announce Type: new Abstract: Text-guided multispectral object detection uses text semantics to guide semantic-aware cross-modal interaction between RGB and IR for more robust percep

SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence

arXiv:2505.17012v3 Announce Type: replace-cross Abstract: Existing evaluations of multimodal large language models (MLLMs) on spatial intelligence are typically fragmented and limited in scope. In thi

VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG

arXiv:2604.05418v2 Announce Type: replace-cross Abstract: Scaling multimodal large language models (MLLMs) to long videos is constrained by limited context windows. While retrieval-augmented generatio

Vision-Language-Action Model, Robustness, Multi-modal Learning, Robot Manipulation

model-releasesarxiv-cs-ro

arXiv:2604.10055v1 Announce Type: new Abstract: Despite their strong performance in embodied tasks, recent Vision-Language-Action (VLA) models remain highly fragile under multimodal perturbations, whe

You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass

arXiv:2604.10966v1 Announce Type: cross Abstract: We present a discriminative multimodal reward model that scores all candidate responses in a single forward pass. Conventional discriminative reward m

3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding

arXiv:2604.08645v1 Announce Type: cross Abstract: Large multimodal models are increasingly used as the reasoning core of embodied agents operating in 3D environments, yet they remain prone to hallucin

researcharxiv-cs-ai

4D-RGPT: Toward Region-level 4D Understanding via Perceptual Distillation

arXiv:2512.17012v3 Announce Type: replace Abstract: Despite advances in Multimodal LLMs (MLLMs), their ability to reason over 3D structures and temporal dynamics remains limited, constrained by weak 4

Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLM

arXiv:2604.06832v2 Announce Type: replace Abstract: Vision-language models (VLMs) predominantly rely on autoregressive decoding, which generates tokens one at a time and fundamentally limits inference

agentsarxiv-cs-cl

How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigms

arXiv:2604.08966v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) have advanced Video Temporal Grounding (VTG), existing methods often couple output paradigms with differe

LADR: Locality-Aware Dynamic Rescue for Efficient Text-to-Image Generation with Diffusion Large Language Models

arXiv:2603.13450v2 Announce Type: replace-cross Abstract: Discrete Diffusion Language Models have emerged as a compelling paradigm for unified multimodal generation, yet their deployment is hindered b

local-aiarxiv-cs-cl

MAG-3D: Multi-Agent Grounded Reasoning for 3D Understanding

arXiv:2604.09167v1 Announce Type: new Abstract: Vision-language models (VLMs) have achieved strong performance in multimodal understanding and reasoning, yet grounded reasoning in 3D scenes remains un

agentsarxiv-cs-cv

VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning

arXiv:2604.09529v1 Announce Type: cross Abstract: Large Vision Language Models (LVLMs) achieve strong multimodal reasoning but frequently exhibit hallucinations and incorrect responses with high certa

researcharxiv-cs-ai

AgriChain Visually Grounded Expert Verified Reasoning for Interpretable Agricultural Vision Language Models

arXiv:2604.07814v1 Announce Type: new Abstract: Accurate and interpretable plant disease diagnosis remains a major challenge for vision-language models (VLMs) in real-world agriculture. We introduce A

Do MLLMs Really Understand Space? A Mathematical Reasoning Evaluation

arXiv:2602.11635v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have achieved strong performance on perception-oriented tasks, yet their ability to perform mathematical sp

EchoTorrent: Towards Swift, Sustained, and Streaming Multi-Modal Video Generation

arXiv:2602.13669v4 Announce Type: replace Abstract: Recent multi-modal video generation models have achieved high visual quality, but their prohibitive latency and limited temporal stability hinder re

Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models

arXiv:2604.01840v2 Announce Type: replace Abstract: While Reinforcement Learning from Verifiable Rewards (RLVR) has advanced reasoning in Large Vision-Language Models (LVLMs), prevailing frameworks su

SpatialMosaic: A Multiview VLM Dataset for Partial Visibility

arXiv:2512.23365v3 Announce Type: replace Abstract: The rapid progress of Multimodal Large Language Models (MLLMs) has unlocked the potential for enhanced 3D scene understanding and spatial reasoning.

Stabilizing Unsupervised Self-Evolution of MLLMs via Continuous Softened Retracing reSampling

arXiv:2604.03647v2 Announce Type: replace-cross Abstract: In the unsupervised self-evolution of Multimodal Large Language Models, the quality of feedback signals during post-training is pivotal for st

researcharxiv-cs-ai

What They Saw, Not Just Where They Looked: Semantic Scanpath Similarity via VLMs and NLP metric

arXiv:2604.08494v1 Announce Type: cross Abstract: Scanpath similarity metrics are central to eye-movement research, yet existing methods predominantly evaluate spatial and temporal alignment while neg

safetyarxiv-cs-cl

DocPilot: Interactive Agent Orchestration and Training for Long Visual Document Understanding

arXiv:2610.21327v1 Announce Type: new Abstract: Multimodal long-document question answering requires models to seek sparse evidence across pages and modalities, understand heterogeneous document conte

agentsarxiv-cs-ai

Tutorials

Full-Duplex Interaction with Speech, Expressions and Gestures

arXiv:2610.21452v1 Announce Type: cross Abstract: Human face-to-face conversation requires coordinating speech, gestures and expressions across both speaking and listening. We present ELLSA 2, an end-

tutorialsarxiv-cs-ai

FutureLift: Lifting Trajectory Proposals into Bidirectional Future Graphs for Motion Forecasting

arXiv:2610.22215v1 Announce Type: new Abstract: Accurate motion prediction is critical for safe autonomous driving, yet even strong forecasters leave residual errors in their multimodal proposals. Exi

agentsarxiv-cs-cv

TimeCap: A Scalable Open-Source Pipeline for Captioning Wearable Health Time-Series Data

model-releasesarxiv-cs-cl

arXiv:2610.21274v1 Announce Type: cross Abstract: Time-series data generated continuously across industrial, environmental, and especially health domains produce multimodal signals whose scale and het