Search: “multimodal”

O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding

arXiv:2605.26584v1 Announce Type: new Abstract: Omnimodal large language models enable unified audio video understanding, but long joint token sequences make inference costly, and existing benchmarks

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants

DGX agent

arXiv:2605.26485v1 Announce Type: cross Abstract: We introduce OmniInteract, a streaming benchmark for real-time omnimodal large language models evaluated through native online inference over audio-vi

model-releasesarxiv-cs-cl

27 May 2026

'PhyWorldBench': A Comprehensive Evaluation of Physical Realism in Text-to-Video Models

arXiv:2507.13428v3 Announce Type: replace-cross Abstract: Video generation models have achieved remarkable progress in creating high-quality, photorealistic content. However, their ability to accurate

PitchBench: Measuring Pitch Hearing in Audio-Language Models

arXiv:2605.26176v1 Announce Type: cross Abstract: Audio-language models (ALMs) are increasingly used in real-world applications that require understanding music, from music tutoring and transcription

Reconstructing Multi-Scale Physical Fields from Extremely Sparse Measurements with an Autoencoder-Diffusion Cascade

arXiv:2512.01572v3 Announce Type: replace-cross Abstract: Extreme sensor sparsity makes full-field reconstruction a fundamentally ill-posed problem in scientific sensing,where the goal is to infer phy

Self-signals Driven Multi-LLM Debate for Efficient and Accurate Reasoning

arXiv:2510.06843v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have exhibited impressive capabilities across diverse application domains. Recent work has explored Multi-LLM Age

The Rescue Effect: Spatio-Semantic Early Exit Bypasses Quantization Collapse in CLIP

arXiv:2605.26415v1 Announce Type: cross Abstract: Deploying Vision-Language Models on resource-constrained hardware typically requires INT8 quantization, but in joint-embedding architectures such as C

Applications

Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance

arXiv:2605.24203v1 Announce Type: new Abstract: Vision-language-action (VLA) models have shown strong potential for generalist robot manipulation, yet they remain limited by insufficient spatial reaso

applicationsarxiv-cs-ro

Agent-Centric Social Trajectory Prediction: A Free Energy Principle Perspective

arXiv:2605.25748v1 Announce Type: new Abstract: Trajectory prediction methods have demonstrated remarkable capabilities in capturing complex motion patterns. However, existing methods rely on global s

AVBench: Human-Aligned and Automated Evaluation Benchmark for Audio-Video Generative Models

arXiv:2605.24652v1 Announce Type: new Abstract: Rapid advances in audio-video (AV) generation have enabled high-fidelity synthesis with synchronized sound, particularly for human-related scenarios inv

Double Triangle Annotation: A Scalable Human-in-the-Loop Framework for High-Precision Historical Document Annotation

model-releasesarxiv-cs-cl

arXiv:2605.25781v1 Announce Type: new Abstract: Evaluating structured-information extraction from historical documents at scale requires high-precision ground-truth annotations, yet traditional manual

FG-CLIP 2: A Bilingual Fine-grained Vision-Language Alignment Model

arXiv:2510.10921v3 Announce Type: replace-cross Abstract: Fine-grained vision-language understanding requires precise alignment between visual content and linguistic descriptions, a capability that re

FoodMonitor: Benchmarking MLLMs for Explainable Compliance Analysis

arXiv:2605.24503v1 Announce Type: cross Abstract: As AI-powered compliance monitoring becomes increasingly important in public governance and industrial safety, the ability to provide verifiable evide

Harnessing AtomisticSkills for Agentic Atomistic Research

arXiv:2605.24002v1 Announce Type: cross Abstract: Computational materials science and chemistry span vast knowledge domains and fractured software ecosystems. Although large language models (LLMs) hav

Language Bias in LVLMs: From In-Depth Analysis to Simple and Effective Mitigation

arXiv:2605.25036v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) extend large language models with visual understanding, but remain vulnerable to hallucination, where outputs are

Learning Transferable Motor Skills for Geometry-Aware Robotic Surface Tasks

arXiv:2605.24881v1 Announce Type: new Abstract: Robotic surface-interaction tasks, such as spray painting or welding, require both accurate geometric planning and precise motion execution. While moder

researcharxiv-cs-ro

MGVQ: Synergizing Multi-dimensional Sensitivity-Aware and Gradient-Hessian Fusion for Vector Quantization

arXiv:2605.24019v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) achieve outstanding performance, yet their huge model size severely hinders deployment on edge devices with limited reso

researcharxiv-cs-lg

Mitigating Object Hallucinations in Vision-Language Models through Region-Aware Attention Recalibration

arXiv:2605.24957v1 Announce Type: new Abstract: The generation of factually incorrect objects, commonly known as object hallucination, remains a persistent challenge in Large Vision-Language Models (L

NeuralTouch: Neural Descriptors for Precise Sim-to-Real Tactile Robot Control

arXiv:2510.20390v2 Announce Type: replace Abstract: Grasping accuracy is a critical prerequisite for precise object manipulation, often requiring careful alignment between the robot hand and object. N

safetyarxiv-cs-ro

RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environments

model-releasesarxiv-cs-ro

arXiv:2509.17057v3 Announce Type: replace Abstract: We present RoboManipBaselines, an open-source software framework for imitation learning research in robotic manipulation. The framework supports the

STaT: Resolving Shape Distortion in Non-Stationary Time Series via Tri-Modal Synergy

arXiv:2605.25943v1 Announce Type: new Abstract: Recent research in time series forecasting frequently investigates the integration of textual and visual modalities with numerical models to better navi

safetyarxiv-cs-lg

Thaka at KSAA-2026 Task 2: Regularized Fine-Tuning for Arabic Speech Diacritization

arXiv:2605.25928v1 Announce Type: new Abstract: We describe the winning system for Task 2 of the KSAA-2026 Shared Task on Arabic Speech Dictation with Automatic Diacritization. The task requires produ

researcharxiv-cs-cl

The Concept Allocation Zone: Tracking How Concepts Form Across Transformer Depth

arXiv:2605.24856v1 Announce Type: cross Abstract: Concept formation in transformer language models is depth-extended, not a single-layer event: concepts emerge gradually across a contiguous region of

Applications

The Meme Is the Message: Generative Memesis and AI Visuals in the 2024 USA Presidential Elections

arXiv:2411.00934v2 Announce Type: replace-cross Abstract: Visual content on social media has become increasingly influential in shaping political discourse and civic engagement, but it also limits par

applicationsarxiv-cs-ai

TopoAlign: Topology-Aware Visual Representation Alignment

arXiv:2605.25541v1 Announce Type: cross Abstract: Neural networks encode inputs as high-dimensional vectors, known as representations, that capture how models process data by encoding task-relevant st

Uni-DPO: A Unified Paradigm for Dynamic Preference Optimization of LLMs

arXiv:2506.10054v4 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) has emerged as a cornerstone of reinforcement learning from human feedback (RLHF) due to its simplicity a

VaaWIT: Visual-Aware Adaptation of Large Language Models for Multilingual Web Image Translation

arXiv:2605.24675v1 Announce Type: cross Abstract: Translating text embedded in Web images is crucial for improving content accessibility and cross-lingual information retrieval, particularly within so

Why We Need World Models for AGI: Where LLMs Fail and How World Models May Outperform

arXiv:2605.23972v1 Announce Type: new Abstract: Large language models achieve strong performance in language generation and knowledge-intensive tasks, yet remain limited in settings requiring causal r

Autonomous Frontier-Based Exploration with VLM Guidance

arXiv:2605.23165v1 Announce Type: cross Abstract: Autonomous robotic exploration of unknown and hazardous environments, a long-standing challenge, can be significantly improved by leveraging the advan

Beyond VLM-Based Rewards: Diffusion-Native Latent Reward Modeling

arXiv:2602.11146v2 Announce Type: replace-cross Abstract: Preference optimization for diffusion and flow-matching models relies on reward functions that are both discriminatively robust and computatio

DDX-TRACE: A Benchmark for Medical Diagnostic Trajectories in VLMs

arXiv:2605.23629v1 Announce Type: new Abstract: Medical diagnosis is not a single prediction from a fully specified vignette. It is a sequential workup: clinicians decide what evidence to obtain, revi

Diffusion and Flow Matching Models for Tabular Data: A Survey

arXiv:2502.17119v2 Announce Type: replace-cross Abstract: Deep generative models have made rapid progress in image, text, audio, and video generation, and are increasingly being applied to structured

Diffusion-based Denoising Beats Vanilla Score Matching in Parameter Estimation: A Theoretical Explanation

model-releasesarxiv-cs-lg

arXiv:2605.22950v1 Announce Type: cross Abstract: Score matching is an alternative to maximum likelihood estimation when the normalizing constant is unknown or too costly to evaluate. However, vanilla

LangFlash: Feed-forward 3D Language Gaussian Splatting from Sparse Unposed Images

arXiv:2605.23287v1 Announce Type: new Abstract: We present LangFlash, a feed-forward framework for 3D Language Gaussian Splatting that reconstructs 3D scenes parameterized by Gaussian primitives enric

researcharxiv-cs-cv

MirrorCheck: Efficient Adversarial Defense for Vision-Language Models

arXiv:2406.09250v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly susceptible to sophisticated adversarial attacks, including adaptive strategies specifically de

Local Ai

PathNavigate: A Training-Free Pathology Agent with Surprise-Guided Scan and Shared Slide Memory for Whole-Slide Image VQA

arXiv:2605.23559v1 Announce Type: cross Abstract: Whole-slide image visual question answering (WSI-VQA) frames pathology as an extreme-context search problem: to answer a free-form clinical query, a s

local-aiarxiv-cs-ai

Scaling-Aware Adapter for Structure-Grounded LLM Reasoning

arXiv:2602.02780v3 Announce Type: replace Abstract: Large language models (LLMs) are enabling reasoning over 2D and 3D structures, yet existing methods remain modality-specific and typically compress

Using Ensemble Diffusion to Estimate Uncertainty for End-to-End Autonomous Driving

arXiv:2506.00560v2 Announce Type: replace-cross Abstract: End-to-end planning systems for autonomous driving are rapidly improving, especially in closed-loop simulation environments like CARLA. Many s

Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability

model-releasesarxiv-cs-lg

arXiv:2605.22168v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) map complex visual inputs to semantic spaces, but interpreting the cross-modal reasoning of VLMs currently relies on pos

23 May 2026

On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs

model-releasesarxiv-cs-lg

arXiv:2602.12506v3 Announce Type: replace Abstract: Reinforcement learning (RL) finetuning has become a key technique for enhancing large language models (LLMs) on reasoning-intensive tasks, motivatin

23 May 2026

Cambrian-P: Pose-Grounded Video Understanding

arXiv:2605.22819v1 Announce Type: new Abstract: Camera pose matters. The position and orientation of each viewpoint define a shared spatial coordinate frame that relates observations across video fram

researcharxiv-cs-cv

Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Large Vision-Language Models

arXiv:2505.16416v3 Announce Type: replace Abstract: Rotary Position Embedding (RoPE) is widely adopted in large language models, but when applied to vision-language models (VLMs) it couples text and i

safetyarxiv-cs-cv

FashionLens: Toward Versatile Fashion Image Retrieval via Task-Adaptive Learning

arXiv:2605.22552v1 Announce Type: new Abstract: Fashion image retrieval is a cornerstone of modern e-commerce systems. A unified framework that supports diverse query formats and search intentions is

From Recognition to Reasoning: Benchmarking and Enhancing MLLMs on Real-World Receipt Document Understanding

arXiv:2605.22413v1 Announce Type: new Abstract: Extracting structured information from visual documents (Visual Information Extraction, VIE) is a cornerstone of business automation. While recent Multi

GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation

arXiv:2605.22036v1 Announce Type: new Abstract: Despite significant progress in Vision-Language Navigation (VLN), existing approaches still rely on dense RGB videos that produce excessive patch tokens

agentsarxiv-cs-cv

GLeVE: Graph-Guided Lesion Grounding with Proposal Verification in 3D CT

arXiv:2605.22619v1 Announce Type: new Abstract: Grounding radiology report descriptions to 3D CT volumes is essential for verifiable clinical interpretation, yet remains challenging due to the semanti

safetyarxiv-cs-cv

How can reasoning capability empower the AI copilot robot in endoscopic surgery

arXiv:2605.22322v1 Announce Type: new Abstract: Reasoning capability has significantly advanced complex logical inference and robotic decision-making in general domains. However, its potential in the

safetyarxiv-cs-ro

How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing

arXiv:2602.01851v2 Announce Type: replace Abstract: Recent generative models have achieved remarkable progress in image editing. However, existing systems and benchmarks remain largely text-guided. In