Search: “multimodal”

Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System

arXiv:2608.11738v1 Announce Type: cross Abstract: Multimodal Large Language Model (MLLM)-based UAV aerial image understanding and reasoning is essential for aerial intelligence yet poses distinct chal

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization

DGX agent

arXiv:2608.12158v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have made rapid progress, yet they still exhibit object hallucination, generating plausible but incorrect descr

researcharxiv-cs-cv

13 Aug 2026

Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams

arXiv:2608.12262v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have been growing the capability for scientific writing and collaboration. For example, OpenAI Prism is a fre

Local Ai

ForgeryVCR: Visual-Centric Reasoning via Efficient Forensic Tools in MLLMs for Image Forgery Detection and Localization

arXiv:2602.14098v2 Announce Type: replace Abstract: Existing Multimodal Large Language Models (MLLMs) for image forgery detection and localization predominantly operate under a text-centric Chain-of-T

local-aiarxiv-cs-cv

GeoBridge: Decoupled Semantic Conditioning for Generative Image Geolocalization

arXiv:2608.11838v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have advanced image geolocalization mainly by improving how they reason about geographic cues. How that reasoni

GUIDE: Governed Unified Intelligence for Document-to-Artifact Generation in Enterprise Settings

arXiv:2608.12133v1 Announce Type: new Abstract: Enterprise guideline documents are heterogeneous and multimodal, combining narrative text, complex tables, and embedded images. Existing LLM and VLM sys

Self-Evolving Code-with-Image Reasoning

arXiv:2608.11292v1 Announce Type: new Abstract: Multimodal models increasingly reach for tools when solving visual tasks (crop, zoom, rotate, brighten), a paradigm known as thinking-with-images. The c

Local Ai

JEPA-WAM: Stage-Level Joint-Embedding Prediction for World-Action Models in Robot Manipulation

arXiv:2608.10780v1 Announce Type: new Abstract: Generalist robot policies aim to map multimodal observations and linguistic task instructions to actions across diverse tasks. However, existing methods

local-aiarxiv-cs-ro

12 Aug 2026

Order Matters: LVLMs as Judges for Temporal Reasoning in Image Sequences

arXiv:2608.10908v1 Announce Type: cross Abstract: As generative multimedia evolves from static image synthesis to complex, interleaved visual narratives, a foundational bottleneck has emerged: the jud

researcharxiv-cs-cl

12 Aug 2026

SafeCap: Improving LVLM Safety with Image Captioning Reinforcement Learning

arXiv:2608.10513v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) remain vulnerable to jailbreak attacks that exploit visual inputs to bypass safety alignment inherited from their

12 Aug 2026

ADOPD: Reference-Privileged On-Policy Distillation for MLLM-Based Industrial Anomaly Detection

arXiv:2608.09789v1 Announce Type: new Abstract: Industrial anomaly detection (IAD) requires identifying fine-grained deviations from normal visual patterns. Multimodal large language models (MLLMs) ca

Advantage-Guided Gate: Reshaping Open-Ended Reasoning for Vision-Based Spatial Intelligence

arXiv:2608.07987v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have demonstrated significant potential in complex spatial scene understanding and reasoning tasks. However, th

Beyond Global Editing: Per-Instance Disentangled Subspaces for Training-Free Hallucination Mitigation in LVLMs

arXiv:2608.09344v1 Announce Type: new Abstract: Recent advances in large vision-language models (LVLMs) have enabled powerful multimodal reasoning by integrating visual encoders with large language mo

Tutorials

Did the Grid Erase the Event? EndoClock for Auditing Medical World-Model Pipelines

arXiv:2608.09266v1 Announce Type: new Abstract: Medical world models commonly learn from multimodal recordings synchronized onto a fixed-rate grid. This preprocessing resamples each native stream onto

tutorialsarxiv-cs-cv

Findings of the First Teaching Monster Challenge: A Benchmark of Pedagogical Content Knowledge in AI Agents

arXiv:2608.08852v1 Announce Type: new Abstract: AI agents can now solve problems, answer like subject experts, and generate long-form multimodal content. However, whether they can adapt a lesson to fi

Local Ai

Hallucination-Free GUI Grounding via Regression-Free Layout-Aware Matching

arXiv:2608.09654v1 Announce Type: new Abstract: GUI agents are shifting from metadata-dependent large language models to purely visual multimodal large language models (MLLMs) that operate directly on

local-aiarxiv-cs-ai

Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models

arXiv:2608.09227v1 Announce Type: new Abstract: Omnimodal language models (OLMs) enable unified audio-visual understanding, but processing long joint token sequences makes inference computationally pr

OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories

model-releasesarxiv-cs-cl

arXiv:2608.08557v1 Announce Type: new Abstract: Visual tool use has emerged as a fundamental capability for multimodal agents to actively acquire evidence beyond a fixed image encoding. The prevailing

Performance of large language models in the optical diagnosis of colorectal polyps

arXiv:2608.07543v1 Announce Type: cross Abstract: Background and Study Aims: Accurate optical diagnosis of colorectal polyps guides resection strategy and surveillance, with multimodal large language

Preference Redirection via Attention Concentration: An Attack on Computer Use Agents

arXiv:2604.08005v2 Announce Type: replace Abstract: Advancements in multimodal foundation models have enabled the development of Computer Use Agents (CUAs) capable of autonomously interacting with GUI

agentsarxiv-cs-lg

Right Answer, Wrong Heat: Explanation-Aware Evaluation and Thermal-Grounded Feedback for MLLMs on Infrared Images

arXiv:2608.09145v1 Announce Type: new Abstract: General-purpose multimodal large language models (MLLMs) are increasingly applied to infrared images, where they are commonly scored by answer accuracy

Self-Evolving Neuro-Symbolic Skills for Tool-Augmented Spatial Reasoning

arXiv:2608.07955v1 Announce Type: new Abstract: Large vision-language models have achieved strong performance in multimodal reasoning, but they remain unreliable on fine-grained spatial tasks that dem

SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation

model-releasesarxiv-cs-ro

arXiv:2608.09771v1 Announce Type: new Abstract: Vision-language-action policies rely on large multimodal backbones to jointly perform perception, language conditioning, and action generation at every

The Judge Knows When It Knows: Calibrated Abstention for LLM-Based A/B-Test Prediction

model-releasesarxiv-cs-cl

arXiv:2608.07517v1 Announce Type: cross Abstract: Can a multimodal LLM predict which version of a web page will win a real A/B test from screenshots alone? We report the most complete answer we are aw

ToolVision: Learning When and How to Use Visual Tools with Capability-Aligned Supervision

arXiv:2608.08907v1 Announce Type: cross Abstract: Thinking with images allows a multimodal model to compensate for limited perception by invoking visual tools through code. Yet the prevailing SFT-then

VCU-Bridge: Hierarchical Visual Connotation Understanding via Semantic Bridging

arXiv:2511.18121v2 Announce Type: replace-cross Abstract: While Multimodal Large Language Models (MLLMs) excel on benchmarks, their processing paradigm differs from the human ability to integrate visu

Your VLM Already Knows When: Training-Free Temporal Grounding by Asking Yes or No

arXiv:2608.08315v1 Announce Type: new Abstract: Multimodal LLMs that recognise events reliably still fail to say when they happen. Prompted for timestamps, strong VLMs reach as little as 3.8% R@0.5 on

AVCap: Reinforcing Audio-Video Joint Caption with Detail-Aware Reward

arXiv:2608.06930v1 Announce Type: new Abstract: Detailed audio-video joint captioning is essential for multimodal video understanding and generation. However, prior works are constrained by three main

CAS2UML: A Handwritten Sketch-to-PlantUML Dataset for Class and Activity Diagrams

arXiv:2608.07036v1 Announce Type: cross Abstract: Automated UML generation from sketches and images is gaining renewed attention with the rise of large language models and multimodal AI. However, repr

Debias in Text, Believe Your Eyes: Text-Anchored Cross-Modal Transfer for Visual Counter-Commonsense Reasoning

arXiv:2608.06938v1 Announce Type: cross Abstract: The visual reasoning ability of multimodal large language models (MLLMs) is crucial for downstream applications, particularly counter-commonsense reas

researcharxiv-cs-ai

I Seek You in Videos: Identity-Conditioned Queries for Person-Centric Video Reasoning

arXiv:2608.07417v1 Announce Type: cross Abstract: Real-world video reasoning often involves multimodal, multi-source inputs, whereas existing video reasoning tasks typically assume a simplified video-

LMM Modality Transfer: A Pre-requisite for Autonomous GIS Agents

arXiv:2608.06948v1 Announce Type: new Abstract: AI models are becoming increasingly adept at understanding and processing spatial information, thereby facilitating agentic problem-solving in spatial t

ReGraph: Learning to Generate Recipe Graphs from Food Images

arXiv:2608.06917v1 Announce Type: new Abstract: Recent Large Multimodal Models (LMMs) have achieved impressive performance in recipe generation from food images.However, cooking is a structured transf

researcharxiv-cs-ai

RoRA: Role-Oriented Regional Allocation for Visual Token Pruning in MLLMs

arXiv:2608.07088v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) encode images as long visual token sequences, making prefilling and KV-cache storage expensive. Existing trai

Toward a Causal Data Management Ecosystem for Decision Making and Agentic AI

arXiv:2608.07214v1 Announce Type: cross Abstract: Modern AI is no longer a single model but an ecosystem: classical ML predictors, deep and multimodal models, large language models, and agents, each t

A Six-Dimensional Taxonomy of Post-Training Adaptation Techniques with Applications in AI Governance

model-releasesarxiv-cs-lg

arXiv:2608.06246v1 Announce Type: new Abstract: Post-training adaptation has become central to modern machine learning practice and includes techniques such as retraining, fine-tuning, parameter-effic

Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs

arXiv:2608.05592v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved strong progress in video understanding, yet it remains challenging because the token limitation m

CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation

arXiv:2603.08652v2 Announce Type: replace Abstract: Recent advancements in Unified Multimodal Models (UMMs) have significantly advanced text-to-image (T2I) generation, particularly through the integra

researcharxiv-cs-ai

Decoupling Perception from Description: Computation-Grounded Representation Alignment between Multivariate Time Series and Language

model-releasesarxiv-cs-lg

arXiv:2608.05238v1 Announce Type: new Abstract: Training multimodal models to align time series with language runs into a self-supervision trap. The usual recipe asks an LLM to read a series and write

DistMedVL: Distributional Vision-Language Alignment for Uncertainty-Aware Medical Image Segmentation

arXiv:2608.05683v1 Announce Type: cross Abstract: Cross-modal alignment of visual and textual representations is fundamental to multimodal medical image understanding, yet remains hindered by uncertai

Nonvisual Classification of Ground-Condition by Artificial Proprioception in an Amoeba-Inspired Autonomous Walking Robot

arXiv:2608.05684v1 Announce Type: cross Abstract: Nonvisual classification of ground condition based on a multimodal sensing approach was investigated for an amoeba-inspired autonomous walking robot.

One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding

arXiv:2608.05707v1 Announce Type: new Abstract: Frame selection is essential for applying Large Multimodal Models (LMMs) to long videos due to severe frame redundancy and limited context windows. Sinc

StepReflect: Structured UI Transition Reflection for Mobile GUI Agents

arXiv:2608.05587v1 Announce Type: new Abstract: Autonomous mobile GUI agents require accurate action reflection for reliable long-horizon execution. Existing approaches rely on open-ended multimodal r

The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images

arXiv:2608.06270v1 Announce Type: new Abstract: The 'thinking-with-images' paradigm equips multimodal LLMs with active visual operations such as crop-and-zoom. However, models using these operations o

Breaking the Curse ofMultilinguality inMany-to-Many Speech-to-Text Translation via a Resource-AwareMixture of Speech Encoders

arXiv:2608.04586v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have achieved significant success in speech-to-text translation (S2TT). However, when processing multilingual

Exo2EgoPose: Leveraging Exocentric Demonstrations for Vision-Language guided Egocentric 3D Hand Pose Forecasting

arXiv:2607.15890v2 Announce Type: replace Abstract: Perceiving multimodal cues and forecasting fine-grained actions from an egocentric (Ego) perspective is vital for applications like robot manipulati

MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understanding

arXiv:2608.04587v1 Announce Type: new Abstract: Long-form video understanding requires locating sparse, question-relevant evidence in long, multimodal videos. Real-world video distributions differ in

agentsarxiv-cs-cv

Reading Between the Frames: Interpreting Implicit and Non-literal Meaning in Social Media Videos

model-releasesarxiv-cs-cl

arXiv:2608.04939v1 Announce Type: new Abstract: Social media videos often communicate meanings that go beyond their visible actions, captions, or speech. A mundane clip may become humorous, ironic, or