Search: “multimodal”

DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cues

arXiv:2606.26602v1 Announce Type: new Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated impressive fine-grained perception capabilities. However, existing ben

26 Jun 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agents

Empowering GUI Agents via Autonomous Experience Exploration and Hindsight Experience Utilization for Task Planning

DGX agent

arXiv:2606.27330v1 Announce Type: cross Abstract: Multimodal web agents can assist humans in operating repetitive GUI tasks, where effective task planning is essential for decomposing complex tasks in

agentsarxiv-cs-ai

26 Jun 2026

MetaboNet-Bench: A Multi-modal Benchmark for Glucose Forecasting in Type 1 Diabetes

model-releasesarxiv-cs-lg

arXiv:2606.18640v2 Announce Type: replace Abstract: Glucose forecasting algorithms are an important aspect of glycemic control management in type 1 diabetes. So far, the research community has develop

26 Jun 2026

Staying VIGILant: Mitigating Visual Laziness via Counterfactual Visual Alignment in MLLMs

arXiv:2606.26387v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) extend large language models (LLMs) with visual perception, enabling joint reasoning over images and text. De

safetyarxiv-cs-cl

26 Jun 2026

AMVICC: A Novel Benchmark for Cross-Modal Failure Mode Profiling for VLMs and IGMs

arXiv:2601.17037v2 Announce Type: replace Abstract: We investigate visual reasoning limitations of both multimodal large language models (MLLMs) and image generation models (IGMs) by creating a novel

Curvature-Guided Mixing for MLLM Adaptation

arXiv:2606.24963v1 Announce Type: new Abstract: Fine-tuning Multimodal Large Language Models (MLLMs) on specialized tasks often leads to catastrophic forgetting of their general capabilities. Existing

Falcon: Functional Assembly and Language for Compositional Reasoning in X-ray

arXiv:2606.25701v1 Announce Type: new Abstract: Conventional vision-language models are largely object-centric, focusing on detecting and describing individual entities. In safety-critical X-ray bagga

Position: Reasoning After Perception Means Reasoning Without Vision

arXiv:2507.16863v2 Announce Type: replace-cross Abstract: A common belief in multimodal research is that the perceptual weaknesses of vision--language models can be compensated by stronger language re

researcharxiv-cs-cl

Speech Codec Probing from Semantic and Phonetic Perspectives

model-releasesarxiv-cs-cl

arXiv:2603.10371v2 Announce Type: replace-cross Abstract: Speech tokenizers are essential for connecting speech to large language models (LLMs) in multimodal systems. Speech tokenizers are expected to

Towards a Dynamic and Fixed-budget Memory Bank for Efficient Streaming Video Understanding

arXiv:2606.25658v1 Announce Type: new Abstract: Currently, streaming video understanding is still a daunting task for existing multimodal large language models (MLLMs). Its difficulties not only lie i

TriViewBench: Controlled Complexity Scaling for Multi-View Structural Reasoning in MLLMs

arXiv:2606.26029v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) demonstrate strong performance on standard visual question answering benchmarks, yet their scalability under co

V-Zero: Answer-Label-Free On-Policy Distillation with Contrastive Evidence Gating for Fine-Grained Visual Reasoning

arXiv:2606.25319v1 Announce Type: new Abstract: Fine-grained visual reasoning requires multimodal large language models (MLLMs) to identify task-relevant visual evidence and ground their reasoning in

Variational Inference via Entropic Transport Descent

arXiv:2606.25265v1 Announce Type: new Abstract: Particle-based variational inference (ParVI) methods approximate an intractable target distribution by evolving an ensemble of interacting samples. Exis

researcharxiv-cs-lg

When Multi-Sensor Fusion Fails to Generalize: Cattle Posture Classification Under Animal-Level and Temporal Distribution Shift

model-releasesarxiv-cs-lg

arXiv:2606.24986v1 Announce Type: new Abstract: Automated cattle posture-classification systems frequently report near-perfect accuracy, yet their robustness under realistic deployment conditions rema

ActiveScope: Actively Seeking and Correcting Perception for MLLMs

arXiv:2606.24292v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated impressive vision-language understanding, yet still struggle with fine-grained perception in

local-aiarxiv-cs-cv

Agentic Collaborative Cognition for Zero-Shot 3D Understanding

arXiv:2606.24649v1 Announce Type: new Abstract: Recent advancements have explored agentic zero-shot 3D understanding by reformulating it as video keyframe understanding with Multimodal Large Language

An LMM for Precisely Grounding Elements in Documents

arXiv:2606.24118v1 Announce Type: new Abstract: Visual grounding in documents is a crucial ability for Large Multimodal Models (LMMs) in areas such as document understanding, deep research and documen

local-aiarxiv-cs-cv

EchoFoley: Event-Centric Hierarchical Control for Video Grounded Creative Sound Generation

arXiv:2512.24731v2 Announce Type: replace Abstract: Sound effects build an essential layer of multimodal storytelling, shaping the emotional atmosphere and the narrative semantics of videos. Despite r

Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control

arXiv:2606.24208v1 Announce Type: new Abstract: Diffusion models sample effectively from high-dimensional, multimodal distributions, but their outputs may violate deployment constraints. For task-spac

researcharxiv-cs-ro

UniDrive: A Unified Vision-Language and Grounding Framework for Interpretable Risk Understanding in Autonomous Driving

model-releasesarxiv-cs-ai

arXiv:2606.24759v1 Announce Type: cross Abstract: Recent multimodal large language models (MLLMs) have shown strong potential for autonomous driving scene understanding, yet existing methods still fac

UniTranslator: A Unified Multi-modal Framework for End-to-end In-Image Machine Translation

arXiv:2606.24333v1 Announce Type: new Abstract: In-Image Machine Translation (IIMT) aims to translate scene text in an image and render the translated text back into the original regions while preserv

Verifiable Foundation Models for Robot Safety

arXiv:2606.23754v1 Announce Type: cross Abstract: Deploying foundation models for robot control raises a central challenge: the expressive power that enables rich, multimodal perception also makes the

safetyarxiv-cs-lg

Applications

ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering

arXiv:2606.24602v1 Announce Type: new Abstract: Despite remarkable progress in multimodal understanding, current MLLMs still exhibit limitations in video text understanding, particularly when semantic

applicationsarxiv-cs-cv

AEF-Econ: Toward Plug-and-Play Socioeconomic Foundation Embeddings from AlphaEarth for Urban Remote Sensing

arXiv:2606.20697v1 Announce Type: new Abstract: AlphaEarth Foundations (AEF) unify global remote sensing foundation embeddings through multimodal self-supervised learning, but their pretraining focuse

AIR: Adaptive Interleaved Reasoning with Code in MLLMs

arXiv:2606.23678v1 Announce Type: new Abstract: Following the paradigm shift initiated by OpenAI o3, interleaved reasoning with code to enhance multimodal large language models (MLLMs) has become a pi

ASCII Art Turns LLMs into VLA Controllers

arXiv:2606.21470v1 Announce Type: cross Abstract: Vision--Language--Action (VLA) controllers are often built by extending vision--language models (VLMs) with action supervision, relying on multimodal

Attention at Rest Stays at Rest: Breaking Visual Inertia for Cognitive Hallucination Mitigation

arXiv:2604.01989v3 Announce Type: replace Abstract: Like a body at rest that stays at rest, we find that visual attention in multimodal large language models (MLLMs) exhibits pronounced inertia, remai

Beyond Flat Labels: Level-Restricted Contrastive Learning for Hierarchical Fine-Grained Vision Classification

arXiv:2606.21838v1 Announce Type: new Abstract: Multimodal contrastive learning has enabled zero-shot visual classification by aligning images with textual categories. However, in hierarchically struc

Tutorials

BoxCtrl: 3D-Aware Visual Prompting for Geometric Image Editing

arXiv:2606.23270v1 Announce Type: new Abstract: As instruction-based editing models and multimodal large language models advance, diverse image editing tasks have become feasible. However, achieving p

tutorialsarxiv-cs-cv

ENVS: Environment-Native Verified Search for Long-Horizon GUI Agents

arXiv:2606.22948v1 Announce Type: cross Abstract: As multimodal agents move from interface understanding to real software control, successful trajectory discovery in live desktop environments becomes

Evaluating and Enhancing Negation Comprehension in Remote Sensing MLLMs

arXiv:2606.20177v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in various Remote Sensing (RS) tasks. However, their ability to compre

Evaluation of Medical Vision Language Models HuluMed and MedGemma, and general purpose chatbots Gemma 3, ChatGPT Plus, and Claude Pro on real previously unseen wound images

arXiv:2606.20723v1 Announce Type: new Abstract: Chronic wound assessment remains a clinically challenging task that requires accurate interpretation of wound morphology, tissue composition, vascular c

Faithful Grounded Visual Reasoning via Learned Proxy-Tokens

arXiv:2606.23354v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable success in Visual Question Answering (VQA), yet their 'black-box' nature hinders deplo

FOCA: Future-Oriented Conditioning for Data-Efficient Vision-Language-Action Adaptation

arXiv:2606.20867v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models enable general-purpose robotic control via large-scale multimodal pretraining, yet their effectiveness under few-sho

Mimic Human Cognition, Master Multi-Image Reasoning: A Meta-Action Framework for Enhanced Visual Understanding

arXiv:2601.07298v2 Announce Type: replace Abstract: While Multimodal Large Language Models (MLLMs) excel at single-image understanding, they exhibit significantly degraded performance in multi-image r

MIRAGE: Stealthy Visual Prompt Injection for Vulnerability Detection in Web Agents

arXiv:2606.20717v1 Announce Type: new Abstract: Multimodal Large Language Model (MLLM)-based web agents provide practical, high-precision solutions for visual browser automation; however, they inheren

local-aiarxiv-cs-cv

OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMs

arXiv:2606.22617v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable performance on 2D visual tasks, yet enhancing their spatial intelligence for real-worl

OphthaDT: Generative Digital Twins for Forecasting Visual Acuity Trajectories in Ophthalmology

arXiv:2606.22101v1 Announce Type: cross Abstract: Precision medicine in ophthalmology requires accurate longitudinal predictions, but the fragmented nature of multimodal clinical data remains a barrie

SPARC: A Multi-Agent System for Electrical Circuit Question Answering

arXiv:2606.20643v1 Announce Type: cross Abstract: Electrical circuit diagram QA tasks require complex mathematical reasoning, which remains challenging for multimodal LLMs. We present SPARC, a multi-a

Temporal Logic Guidance for Action-Only Diffusion Policies with World Models

arXiv:2606.22729v1 Announce Type: new Abstract: Diffusion policies enable multimodal robot behavior but offer limited ability to choose among behavior modes at inference time, even though such control

safetyarxiv-cs-ro

VideoLatent: Video-Language Learning via Latent Self-Forcing

arXiv:2606.22870v1 Announce Type: new Abstract: Recent advancements in chain-of-thought (CoT) reasoning have shown promise in enhancing video understanding and reasoning capabilities of multimodal lar

Adapting Vision-Language Models from Iconic to Inclusive for Multi-Label Recognition Without Labels

arXiv:2606.11626v1 Announce Type: new Abstract: Understanding multi-label images remains a challenging task in computer vision. With the rapid progress of vision-language multimodal learning, vision-l

ConsistencyPlanner: Real-time Planning with Fast-Sampling Consistency Models

arXiv:2606.11569v1 Announce Type: cross Abstract: Closed-loop planning in complex, real-world driving scenarios presents a critical challenge for autonomous driving systems. While traditional rule-bas

safetyarxiv-cs-ai

Applications

Diffusion-based Cumulative Adversarial Purification for Vision Language Models

arXiv:2506.03933v2 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) have shown remarkable capabilities in multimodal understanding, yet their susceptibility to adversarial perturba

applicationsarxiv-cs-ai

From Correspondence to Actions: Human-Like Multi-Image Spatial Reasoning in Multi-modal Large Language Models

arXiv:2602.08735v3 Announce Type: replace Abstract: While multimodal large language models (MLLMs) have made substantial progress in single-image spatial reasoning, multi-image spatial reasoning, whic

Intelligent Automation for Embodied Benchmark Construction: Pipelines, Embodiments, Simulators, and Trends

model-releasesarxiv-cs-ai

arXiv:2606.12207v1 Announce Type: cross Abstract: Embodied intelligence now spans navigation, household assistance, manipulation, autonomous driving, aerial agents, and multimodal large-model control.

MODF-SIR: A Multi-agent Omni-modal Distilled Framework for Social Intelligence Reasoning

arXiv:2606.12018v1 Announce Type: new Abstract: We propose a multi-agent collaborative framework built upon a lightweight Multimodal Large Language Model (MLLM), specifically designed for social intel

local-aiarxiv-cs-ai

Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoning

arXiv:2606.11719v1 Announce Type: cross Abstract: Spatial reasoning remains a persistent challenge for multimodal large language models (MLLMs). Existing approaches largely rely on large-scale, static

researcharxiv-cs-ai