Search: “multimodal”

ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescue

model-releasesarxiv-cs-ro

arXiv:2605.01371v1 Announce Type: new Abstract: The rapid advancement of Multimodal Large Language Models (MLLMs) has empowered Unmanned Aerial Vehicle (UAV) with exceptional capabilities in spatial r

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs

DGX agent

arXiv:2601.22709v3 Announce Type: replace Abstract: Vision-Language Models (VLMs) achieve strong multimodal performance but are costly to deploy, and post-training quantization often causes significan

model-releasesarxiv-cs-cv

5 May 2026

High-Speed, Scalable Sensor Readout for Dexterous Robotic Hands via Shift-Register Multiplexing

arXiv:2605.01434v1 Announce Type: new Abstract: Dexterous robotic hands require high-speed multimodal sensing across many degrees of freedom, yet existing readout architectures often impose trade-offs

researcharxiv-cs-ro

Injecting Distributional Awareness into MLLMs via Reinforcement Learning for Deep Imbalanced Regression

arXiv:2605.01402v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) struggle with numerical regression under long-tailed target distributions. Token-level supervised fine-tuning (

safetyarxiv-cs-cl

Omni-NegCLIP: Enhancing CLIP with Front-Layer Contrastive Fine-Tuning for Comprehensive Negation Understanding

arXiv:2603.29258v2 Announce Type: replace Abstract: Vision-Language Models (VLMs) have demonstrated strong capabilities across a wide range of multimodal tasks. However, recent studies have shown that

researcharxiv-cs-cv

Seeing the Scene Matters: Revealing Forgetting in Video Understanding Models with a Scene-Aware Long-Video Benchmark

arXiv:2603.27259v2 Announce Type: replace Abstract: Long video understanding (LVU) remains a core challenge in multimodal learning. Although recent vision-language models (VLMs) have made notable prog

Sound Source Localization for Spatial Mapping of Surgical Actions in Dynamic Scenes

arXiv:2510.24332v3 Announce Type: replace-cross Abstract: Purpose: Surgical scene understanding is key to advancing computer-aided and intelligent surgical systems. Current approaches predominantly re

agentsarxiv-cs-cv

Video Active Perception: Effective Inference-Time Long-Form Video Understanding with Vision-Language Models

arXiv:2605.01662v1 Announce Type: new Abstract: Large vision-language models (VLMs) have advanced multimodal tasks such as video question answering (QA). However, VLMs face the challenge of selecting

Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs

model-releasesarxiv-cs-lg

arXiv:2605.02735v1 Announce Type: new Abstract: Continuous latent-space reasoning offers a compact alternative to textual chain-of-thought for multimodal models, enabling high-dimensional visual evide

X2SAM: Any Segmentation in Images and Videos

arXiv:2605.00891v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong image-level visual understanding and reasoning, yet their pixel-level perception acros

Being-H0.7: A Latent World-Action Model from Egocentric Videos

arXiv:2605.00078v1 Announce Type: cross Abstract: Visual-Language-Action models (VLAs) have advanced generalist robot control by mapping multimodal observations and language instructions directly to a

applicationsarxiv-cs-cv

FreeRet: MLLMs as Training-Free Retrievers

arXiv:2509.24621v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) are emerging as versatile foundations for mixed-modality retrieval. Yet, they often require heavy post-hoc

safetyarxiv-cs-cv

Group Cognition Learning: Making Everything Better Through Governed Two-Stage Agents Collaboration

arXiv:2605.00370v1 Announce Type: new Abstract: Centralized multimodal learning commonly compresses language, acoustic, and visual signals into a single fused representation for prediction. While effe

agentsarxiv-cs-lg

MMAudio-LABEL: Audio Event Labeling via Audio Generation for Silent Video

arXiv:2605.00495v1 Announce Type: cross Abstract: Recent advances in multimodal generation have enabled high-quality audio generation from silent videos. Practical applications, such as sound producti

applicationsarxiv-cs-cv

Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs

arXiv:2605.00814v1 Announce Type: new Abstract: While autoregressive Large Vision-Language Models (LVLMs) demonstrate remarkable proficiency in multimodal tasks, they face a 'Visual Signal Dilution' p

Thinking with Geometry: Active Geometry Integration for Spatial Reasoning

arXiv:2602.06037v4 Announce Type: replace Abstract: Recent progress in spatial reasoning with Multimodal Large Language Models (MLLMs) increasingly leverages geometric priors from 3D encoders. However

agentsarxiv-cs-cv

VideoDetective: Clue Hunting via both Extrinsic Query and Intrinsic Relevance for Long Video Understanding

arXiv:2603.22285v2 Announce Type: replace Abstract: Long video understanding remains challenging for multimodal large language models (MLLMs) due to limited context windows, which necessitate identify

researcharxiv-cs-cv

A Pattern Language for Resilient Visual Agents

arXiv:2604.28001v1 Announce Type: new Abstract: Integrating multimodal foundation models into enterprise ecosystems presents a fundamental software architecture challenge. Architects must balance comp

applicationsarxiv-cs-ai

AutoSurfer -- Teaching Web Agents through Comprehensive Surfing, Learning, and Modeling

arXiv:2604.27253v1 Announce Type: new Abstract: Recent advances in multimodal large language models (LLMs) have revolutionized web agents that can automate complex tasks on websites. However, their ac

Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMs

arXiv:2506.07180v3 Announce Type: replace-cross Abstract: As video large language models (Video-LLMs) become increasingly integrated into real-world applications that demand grounded multimodal reason

OR-VSKC: Resolving Visual-Semantic Knowledge Conflicts in Operating Rooms with Synthetic Data-Guided Alignment

arXiv:2506.22500v2 Announce Type: replace-cross Abstract: Automated identification of surgical safety risks is critical for improving patient outcomes; however, Multimodal Large Language Models (MLLMs

Delineating Knowledge Boundaries for Honest Large Vision-Language Models

arXiv:2604.26419v1 Announce Type: cross Abstract: Large Vision-Language Models (VLMs) have achieved remarkable multimodal performance yet remain prone to factual hallucinations, particularly in long-t

researcharxiv-cs-ai

FlowS: One-Step Motion Prediction via Local Transport Conditioning

model-releasesarxiv-cs-ro

arXiv:2604.26065v1 Announce Type: new Abstract: Generative motion prediction must satisfy three simultaneous requirements for real-world autonomy: high accuracy, diverse multimodal futures, and strict

Glance-or-Gaze: Incentivizing LMMs to Adaptively Focus Search via Reinforcement Learning

arXiv:2601.13942v2 Announce Type: replace-cross Abstract: Large Multimodal Models (LMMs) have achieved remarkable success in visual understanding, yet they struggle with knowledge-intensive queries in

Thinking with Drafting: Optical Decompression via Logical Reconstruction

arXiv:2602.11731v2 Announce Type: replace Abstract: Existing multimodal large language models have achieved high-fidelity visual perception and exploratory visual generation. However, a precision para

Three-Step Nav: A Hierarchical Global-Local Planner for Zero-Shot Vision-and-Language Navigation

arXiv:2604.26946v1 Announce Type: new Abstract: Breakthrough progress in vision-based navigation through unknown environments has been achieved by using multimodal large language models (MLLMs). These

agentsarxiv-cs-cv

Mitigating Coordinate Prediction Bias from Positional Encoding Failures

arXiv:2510.22102v2 Announce Type: replace-cross Abstract: While Multimodal Large Language Models (MLLMs) excel at general vision-language tasks, precise coordinate prediction remains a significant cha

OMHBench: Benchmarking Balanced and Grounded Omni-Modal Multi-Hop Reasoning

arXiv:2508.16198v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have increasingly supported omni-modal processing across text, vision, and speech. However, existing evalua

Local Ai

OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding

arXiv:2604.25276v1 Announce Type: new Abstract: Video Temporal Grounding (VTG), the task of localizing video segments from text queries, struggles in open-world settings due to limited dataset scale a

local-aiarxiv-cs-cv

SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring

arXiv:2604.25855v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) achieve ever-stronger performance on visual-language tasks. Even as traditional visual question answering bench

Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs

arXiv:2604.24395v1 Announce Type: new Abstract: Large Vision-Language Models (LVLMs) frequently suffer from hallucinations. Existing preference learning-based approaches largely rely on proprietary mo

Beyond Cross-Modal Alignment: Measuring and Leveraging Modality Gap in Vision-Language Models

arXiv:2502.14888v4 Announce Type: replace-cross Abstract: The success of vision-language models is primarily attributed to effective alignment across modalities such as vision and language. However, m

Boosting MLLM Spatial Reasoning with Geometrically Referenced 3D Scene Representations

arXiv:2603.08592v2 Announce Type: replace Abstract: While Multimodal Large Language Models (MLLMs) have achieved remarkable success in 2D visual understanding, their ability to reason about 3D space r

Branching Flows: Discrete, Continuous, and Manifold Flow Matching with Splits and Deletions

arXiv:2511.09465v3 Announce Type: replace-cross Abstract: Diffusion and flow matching approaches to generative modeling have shown promise in domains where the state space is continuous, such as image

researcharxiv-cs-lg

DARC-CLIP: Dynamic Adaptive Refinement with Cross-Attention for Meme Understanding

arXiv:2604.23214v1 Announce Type: new Abstract: Memes convey meaning through the interaction of visual and textual signals, often combining humor, irony, and offense in subtle ways. Detecting harmful

Emotion-Conditioned Short-Horizon Human Pose Forecasting with a Lightweight Predictive World Model

arXiv:2604.23532v1 Announce Type: cross Abstract: Short-term human pose prediction plays a crucial role in interactive systems, assistive robots, and emotion-aware human-computer interaction[1-3]. Whi

researcharxiv-cs-ai

Federated Cross-Modal Retrieval with Missing Modalities via Semantic Routing and Adapter Personalization

arXiv:2604.22885v1 Announce Type: cross Abstract: Federated cross-modal retrieval faces severe challenges from heterogeneous client data, particularly non-IID semantic distributions and missing modali

Hybrid JIT-CUDA Graph Optimization for Low-Latency Large Language Model Inference

arXiv:2604.23467v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved strong performance across natural language and multimodal tasks, yet their practical deployment remains con

Quantum Kernel Advantage over Classical Collapse in Medical Foundation Model Embeddings

arXiv:2604.24597v1 Announce Type: cross Abstract: We provide evidence of quantum kernel advantage under noiseless simulation in binary insurance classification on MIMIC-CXR chest radiographs using qua

researcharxiv-cs-ai

Tutorials

Robust Grounding with MLLMs against Occlusion and Small Objects via Language-guided Semantic Cues

arXiv:2604.24036v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) have enhanced grounding capabilities in general scenes, their robustness in crowded scenes remains undere

tutorialsarxiv-cs-cv

StructAlign: Structured Cross-Modal Alignment for Continual Text-to-Video Retrieval

arXiv:2601.20597v2 Announce Type: replace Abstract: Continual Text-to-Video Retrieval (CTVR) is a challenging multimodal continual learning setting, where models must incrementally learn new semantic

UpstreamQA: A Modular Framework for Explicit Reasoning on Video Question Answering Tasks

arXiv:2604.23145v1 Announce Type: cross Abstract: Video Question Answering (VideoQA) demands models that jointly reason over spatial, temporal, and linguistic cues. However, the task's inherent comple

Adapting MLLMs for Nuanced Video Retrieval

arXiv:2512.13511v2 Announce Type: replace Abstract: Our objective is to build an embedding model that captures the nuanced relationship between a search query and candidate videos. We cover three aspe

researcharxiv-cs-cv

CognitiveTwin: Robust Multi-Modal Digital Twins for Predicting Cognitive Decline in Alzheimer's Disease

arXiv:2604.22428v1 Announce Type: new Abstract: Predicting individual cognitive decline in Alzheimer's disease (AD) is difficult due to the heterogeneity of disease progression. Reliable clinical tool

SOLAR-RL: Semi-Online Long-horizon Assignment Reinforcement Learning

arXiv:2604.22558v1 Announce Type: cross Abstract: As Multimodal Large Language Models (MLLMs) mature, GUI agents are evolving from static interactions to complex navigation. While Reinforcement Learni

agentsarxiv-cs-ai

SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments

arXiv:2604.22409v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have advanced static visual--spatial reasoning, yet they often fail to preserve long-horizon spatial coherence

ViFiCon: Vision and Wireless Association Via Self-Supervised Contrastive Learning

arXiv:2210.05513v2 Announce Type: replace Abstract: We introduce ViFiCon, a self-supervised contrastive scheme which learns a cross-modal association between vision and wireless modalities. Specifical

applicationsarxiv-cs-cv

Can MLLMs 'Read' What is Missing?

arXiv:2604.21277v1 Announce Type: new Abstract: We introduce MMTR-Bench, a benchmark designed to evaluate the intrinsic ability of Multimodal Large Language Models (MLLMs) to reconstruct masked text d