Search: “multimodal”

Unifying Scientific Communication: Fine-Grained Correspondence Across Scientific Media

arXiv:2605.05831v2 Announce Type: replace Abstract: The communication of scientific knowledge has become increasingly multimodal, spanning text, visuals, and speech through materials such as research

12 May 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

From Clouds to Hallucinations: Atmospheric Retrieval Hijacking in Remote Sensing Vision-Language RAG

DGX agent

arXiv:2605.07273v1 Announce Type: cross Abstract: Multimodal RAG systems increasingly rely on vision-language retrievers to ground visual queries in external textual evidence. Existing adversarial stu

model-releasesarxiv-cs-ai

11 May 2026

Qwen3-VL-Seg: Unlocking Open-World Referring Segmentation with Vision-Language Grounding

arXiv:2605.07141v1 Announce Type: cross Abstract: Open-world referring segmentation requires grounding unconstrained language expressions to precise pixel-level regions. Existing multimodal large lang

11 May 2026

Bridging Modalities: Joint Synthesis and Registration Framework for Aligning Diffusion MRI with T1-Weighted Images

arXiv:2601.11689v2 Announce Type: replace-cross Abstract: Multimodal image registration between diffusion MRI (dMRI) and T1-weighted (T1w) MRI images is a critical step for aligning diffusion-weighted

7 May 2026

Disentangling Shared and Task-Specific Representations from Multi-Modal Clinical Data

model-releasesarxiv-cs-lg

arXiv:2605.03570v1 Announce Type: new Abstract: Real-world clinical data is inherently multimodal, providing complementary evidence that mirrors the practical necessity of jointly assessing multiple r

6 May 2026

Agents

MM-StanceDet: Retrieval-Augmented Multi-modal Multi-agent Stance Detection

arXiv:2604.27934v1 Announce Type: new Abstract: Multimodal Stance Detection (MSD) is crucial for understanding public discourse, yet effectively fusing text and image, especially with conflicting sign

agentsarxiv-cs-ai

1 May 2026

SpecVQA: A Benchmark for Spectral Understanding and Visual Question Answering in Scientific Images

arXiv:2604.28039v1 Announce Type: new Abstract: Spectra are a prevalent yet highly information-dense form of scientific imagery, presenting substantial challenges to multimodal large language models (

1 May 2026

Agents

EvolvingAgent: Curriculum Self-evolving Agent with Continual World Model for Long-Horizon Tasks

arXiv:2502.05907v3 Announce Type: replace Abstract: Completing Long-Horizon (LH) tasks in open-ended worlds is an important yet difficult problem for embodied agents. Existing approaches suffer from t

agentsarxiv-cs-ro

30 Apr 2026

Learning from Medical Entity Trees: An Entity-Centric Medical Data Engineering Framework for MLLMs

arXiv:2604.25296v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have shown transformative potential in medical applications, yet their performance is hindered by conventional

safetyarxiv-cs-cl

29 Apr 2026

OneThinker: All-in-one Reasoning Model for Image and Video

arXiv:2512.03043v3 Announce Type: replace Abstract: Reinforcement learning (RL) has recently achieved remarkable success in eliciting visual reasoning within Multimodal Large Language Models (MLLMs).

29 Apr 2026

Applications

Au-M-ol: A Unified Model for Medical Audio and Language Understanding

arXiv:2604.23284v1 Announce Type: cross Abstract: In this work, we present Au-M-ol, a novel multimodal architecture that extends Large Language Models (LLMs) with audio processing. It is designed to i

applicationsarxiv-cs-ai

From Skeletons to Pixels: Few-Shot Precise Event Spotting via Representation and Prediction Distillation

arXiv:2604.22839v1 Announce Type: cross Abstract: Precise Event Spotting (PES) is essential in fast-paced sports such as tennis, where fine-grained events occur within very short temporal windows. Acc

researcharxiv-cs-ai

IntentVLM: Open-Vocabulary Intention Recognition through Forward-Inverse Modeling with Video-Language Models

arXiv:2604.24002v1 Announce Type: cross Abstract: Improving the effectiveness of human-robot interaction requires social robots to accurately infer human goals through robust intention understanding.

researcharxiv-cs-ai

Making Dialogue Grounding Data Rich: A Three-Tier Data Synthesis Framework for Generalized Referring Expression Comprehension

arXiv:2512.02791v2 Announce Type: replace Abstract: Dialogue-Based Generalized Referring Expression Comprehension (GREC) requires models to ground the expression and unlimited targets in complex visua

researcharxiv-cs-cl

MIRAGE: A Micro-Interaction Relational Architecture for Grounded Exploration in Multi-Figure Artworks

arXiv:2604.23788v1 Announce Type: new Abstract: Appreciating multi-figure paintings requires understanding how characters relate through subtle cues like gaze alignment, gesture, and spatial arrangeme

BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answering

arXiv:2604.22678v1 Announce Type: new Abstract: A common approach to question answering with retrieval-augmented generation (RAG) is to concatenate documents into a single context and pass it to a lan

researcharxiv-cs-cl

27 Apr 2026

Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning Segmentation

arXiv:2510.19592v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) demonstrate strong video understanding by attending to visual tokens relevant to textual queries. To direct

27 Apr 2026

Balanced Performance Across Artistic Styles： More uniform quality across diverse aesthetic domains, effectively reducing style-dependent qua…

Qwen's latest model improvements focus on achieving more consistent and uniform performance quality across different artistic styles and aesthetic domains, reducing the variability in output quality t

model-releasesqwen--x

25 Apr 2026

ReCAPA: Hierarchical Predictive Correction to Mitigate Cascading Failures

arXiv:2604.21232v1 Announce Type: new Abstract: Vision-Language-Action systems follow instructions to execute multi-step tasks in multimodal environments. Recent VLA approaches typically rely on post-

local-aiarxiv-cs-ai

24 Apr 2026

SGG-R^{rm 3}: From Next-Token Prediction to End-to-End Unbiased Scene Graph Generation

arXiv:2603.07961v3 Announce Type: replace Abstract: Scene Graph Generation (SGG) structures visual scenes as graphs of objects and their relations. While Multimodal Large Language Models (MLLMs) have

24 Apr 2026

Rethinking Reinforcement Fine-Tuning in LVLM: Convergence, Reward Decomposition, and Generalization

arXiv:2604.19857v1 Announce Type: cross Abstract: Reinforcement fine-tuning with verifiable rewards (RLVR) has emerged as a powerful paradigm for equipping large vision-language models (LVLMs) with ag

safetyarxiv-cs-cl

23 Apr 2026

Agents

zero shot Kimi K2.6, go try it out its a good model sir! this is @Kimi_Moonshot running on @togethercompute, @opencode harness prompt below…

zero shot Kimi K2.6, go try it out its a good model sir! this is @Kimi_Moonshot running on @togethercompute, @opencode harness prompt below👇 Media Introducing Kimi K2.6 from @Kimi_Moonshot, a multimod

agentstogether-ai--x

23 Apr 2026

Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Models

arXiv:2601.22737v2 Announce Type: replace Abstract: The robust safety of Vision-Language Large Models (VLLMs) against joint multilingual and multimodal threats remains severely underexplored. Current

22 Apr 2026

BOOKAGENT: Orchestrating Safety-Aware Visual Narratives via Multi-Agent Cognitive Calibration

arXiv:2604.16541v1 Announce Type: new Abstract: Recent advancements in Large Generative Models (LGMs) have revolutionized multi-modal generation. However, generating illustrated storybooks remains an

Emotion Collider: Dual Hyperbolic Mirror Manifolds for Sentiment Recovery via Anti Emotion Reflection

arXiv:2602.16161v3 Announce Type: replace-cross Abstract: Emotional expression underpins natural communication and effective human-computer interaction. We present Emotion Collider (EC-Net), a hyperbo

researcharxiv-cs-cl

End-to-end Listen, Look, Speak and Act

model-releasesarxiv-cs-cl

arXiv:2510.16756v2 Announce Type: replace-cross Abstract: Human interaction is inherently multimodal and full-duplex: we listen while watching, speak while acting, and fluidly adapt to turn-taking and

Geometry-Guided 3D Visual Token Pruning for Video-Language Models

arXiv:2604.18260v1 Announce Type: new Abstract: Multimodal large language models have demonstrated remarkable capabilities in 2D vision, motivating their extension to 3D scene understanding. Recent st

OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models

arXiv:2511.14582v2 Announce Type: replace Abstract: Omnimodal large language models (OmniLLMs) have attracted increasing research attention of late towards unified audio-video understanding. However,

Real-Time Visual Attribution Streaming in Thinking Model

arXiv:2604.16587v1 Announce Type: new Abstract: We present an amortized framework for real-time visual attribution streaming in multimodal thinking models. When these models generate code from a scree

Joint-Centric Dual Contrastive Alignment with Structure-Preserving and Information-Balanced Regularization

arXiv:2604.16247v1 Announce Type: cross Abstract: We propose HILBERT (HIerarchical Long-sequence Balanced Embedding with Reciprocal contrastive Training), a cross-attentive multimodal framework for le

20 Apr 2026

VeRVE: Versatile Retrieval for Videos via Unified Embeddings

arXiv:2601.12193v3 Announce Type: replace Abstract: Modern video retrieval systems are expected to handle diverse tasks ranging from corpus-level retrieval, fine-grained moment localization to flexibl

local-aiarxiv-cs-cv

20 Apr 2026

CaptionQA: Is Your Caption as Useful as the Image Itself?

arXiv:2511.21025v2 Announce Type: replace Abstract: Image captions serve as efficient surrogates for visual content in multimodal systems such as retrieval, recommendation, and multi-step agentic infe

Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models

arXiv:2604.14888v1 Announce Type: new Abstract: Recent advances in vision language models (VLMs) offer reasoning capabilities, yet how these unfold and integrate visual and textual information remains

safetyarxiv-cs-cl

Towards Scalable Lightweight GUI Agents via Multi-role Orchestration

arXiv:2604.13488v1 Announce Type: new Abstract: Autonomous Graphical User Interface (GUI) agents powered by Multimodal Large Language Models (MLLMs) enable digital automation on end-user devices. Whil

V-Reflection: Transforming MLLMs from Passive Observers to Active Interrogators

arXiv:2604.03307v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable success, yet they remain prone to perception-related hallucinations in fine-graine

local-aiarxiv-cs-cv

Character Beyond Speech: Leveraging Role-Playing Evaluation in Audio Large Language Models via Reinforcement Learning

arXiv:2604.13804v1 Announce Type: new Abstract: The rapid evolution of multimodal large models has revolutionized the simulation of diverse characters in speech dialogue systems, enabling a novel inte

safetyarxiv-cs-lg

UMI-3D: Extending Universal Manipulation Interface from Vision-Limited to 3D Spatial Perception

arXiv:2604.14089v1 Announce Type: new Abstract: We present UMI-3D, a multimodal extension of the Universal Manipulation Interface (UMI) for robust and scalable data collection in embodied manipulation

safetyarxiv-cs-ro

ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Body

arXiv:2512.14234v2 Announce Type: replace Abstract: Human communication is inherently multimodal and social: words, prosody, and body language jointly carry intent. Yet most prior systems model human

Visual Self-Fulfilling Alignment: Shaping Safety-Oriented Personas via Threat-Related Images

arXiv:2603.08486v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) face safety misalignment, where visual inputs enable harmful outputs. To address this, existing methods req

Every Picture Tells a Dangerous Story: Memory-Augmented Multi-Agent Jailbreak Attacks on VLMs

arXiv:2604.12616v1 Announce Type: new Abstract: The rapid evolution of Vision-Language Models (VLMs) has catalyzed unprecedented capabilities in artificial intelligence; however, this continuous modal

15 Apr 2026

Why isn't JoyAI Image Edit getting any love?

local-air-stablediffusion

A Reddit thread on r/StableDiffusion questioning why JoyAI Image Edit — a unified multimodal foundation model for image understanding, text-to-image generation, and instruction-guided image editing —

15 Apr 2026

Bridging the RGB-IR Gap: Consensus and Discrepancy Modeling for Text-Guided Multispectral Detection

arXiv:2604.11234v1 Announce Type: new Abstract: Text-guided multispectral object detection uses text semantics to guide semantic-aware cross-modal interaction between RGB and IR for more robust percep

SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence

arXiv:2505.17012v3 Announce Type: replace-cross Abstract: Existing evaluations of multimodal large language models (MLLMs) on spatial intelligence are typically fragmented and limited in scope. In thi

VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG

arXiv:2604.05418v2 Announce Type: replace-cross Abstract: Scaling multimodal large language models (MLLMs) to long videos is constrained by limited context windows. While retrieval-augmented generatio

Vision-Language-Action Model, Robustness, Multi-modal Learning, Robot Manipulation

model-releasesarxiv-cs-ro

arXiv:2604.10055v1 Announce Type: new Abstract: Despite their strong performance in embodied tasks, recent Vision-Language-Action (VLA) models remain highly fragile under multimodal perturbations, whe

You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass

arXiv:2604.10966v1 Announce Type: cross Abstract: We present a discriminative multimodal reward model that scores all candidate responses in a single forward pass. Conventional discriminative reward m

3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding

arXiv:2604.08645v1 Announce Type: cross Abstract: Large multimodal models are increasingly used as the reasoning core of embodied agents operating in 3D environments, yet they remain prone to hallucin

researcharxiv-cs-ai

13 Apr 2026

4D-RGPT: Toward Region-level 4D Understanding via Perceptual Distillation

arXiv:2512.17012v3 Announce Type: replace Abstract: Despite advances in Multimodal LLMs (MLLMs), their ability to reason over 3D structures and temporal dynamics remains limited, constrained by weak 4