Search: “multimodal”

ETHOS: Towards a Modular Ethics Framework for Clinical Multi-Agent Systems

arXiv:2608.15424v1 Announce Type: cross Abstract: The rapid adoption of large language models has enabled the development of clinical multi-agent systems (MAS) capable of integrating multimodal patien

safetyarxiv-cs-ai

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting

DGX agent

arXiv:2608.15310v1 Announce Type: cross Abstract: Multimodal data collected by heterogeneous devices are used for collaborative training, where federated learning (FL) serves as a key paradigm for eff

safetyarxiv-cs-lg

18 Aug 2026

Matched Outcomes, Divergent Gaze: How Foveated MLLMs Search Compared to Humans

arXiv:2608.16514v1 Announce Type: cross Abstract: Human visual search is serial: the fovea must land on a candidate to confirm it, and those landings form a scanpath. Whether multimodal large language

safetyarxiv-cs-ai

MEDR: Query-Independent Frame Selection via Multi-Signal Event Modeling and Dynamic Rescoring

arXiv:2608.15058v1 Announce Type: new Abstract: Frame selection is a fundamental component of multimodal large language models, enabling long videos to be processed under limited visual-token and comp

Modality-Invariant Coarse-to-Fine Retinal Image Registration

arXiv:2608.14829v1 Announce Type: cross Abstract: Retinal image registration is essential for ophthalmic diagnosis, longitudinal disease monitoring, and multimodal retinal image analysis. Existing ret

safetyarxiv-cs-cv

NumerosityVLM: A Cognitively Inspired Benchmark for Interpreting Numerosity Representations in Vision-Language Models

arXiv:2608.15425v1 Announce Type: cross Abstract: Vision-language models (VLMs) achieve strong performance on high-level multimodal tasks, yet numerosity perception, a cognitive ability that emerges i

Physiological World Models for Human State Transitions

arXiv:2608.15309v1 Announce Type: new Abstract: Continuous multimodal sensing now allows human physiology to be observed throughout daily life rather than only during occasional clinical visits. Howev

Planner-Conditioned Diffusion for Coordinated Multi-Agent Exploration

arXiv:2608.16229v1 Announce Type: new Abstract: Coordinated multi-agent exploration requires not only efficient individual coverage but also non-redundant coverage across agents over extended planning

safetyarxiv-cs-ro

SA-GEM: Scale-Adaptive and Geospatial Evidence-Modulated Token Pruning for Efficient Remote Sensing Large Vision-Language Models

arXiv:2608.15075v1 Announce Type: new Abstract: RS-LVLMs have advanced multimodal understanding of Earth observation imagery, yet their performance is fundamentally constrained by high-resolution proc

local-aiarxiv-cs-cv

TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation

arXiv:2608.16765v1 Announce Type: cross Abstract: Despite recent advances in unified multimodal models for multi-reference image generation, existing benchmarks remain organized around predefined task

local-aiarxiv-cs-ai

UniFed-VLM: Federated Instruction Tuning for Vision-Language Models with Multiple Heterogeneity

model-releasesarxiv-cs-lg

arXiv:2608.15516v1 Announce Type: new Abstract: Vision-Language Models (VLMs) have demonstrated strong performance in multimodal understanding and generation. However, fine-tuning of VLMs typically re

Act2Intention: A Benchmark For Developing Active Mobile Agents Through Inferring User Intention from GUI Actions

arXiv:2608.14132v1 Announce Type: cross Abstract: Mobile GUI Agents powered by multimodal large language models (MLLMs) show promise in human-computer intelligence. However, current research primarily

CRAFT: Constrained Reward via Attention Fine-Tuning for Subject Personalization without Composed Targets

arXiv:2608.14403v1 Announce Type: new Abstract: Subject-driven image personalization---generating new images that preserve the identity of one or several reference subjects in novel scenes---is a foun

Data-driven techniques for translational neuroscience and personalized neuro-health

arXiv:2608.13749v1 Announce Type: cross Abstract: Neurodegenexrative diseases such as Alzheimer's disease and Parkinson's disease are diagnosed most reliably only after substantial, often irreversible

researcharxiv-cs-ai

EchoChange: A Diffusion Language Model with Dual Pass Remasking for Factual Remote Sensing Disaster Change Captioning

arXiv:2608.01856v2 Announce Type: replace Abstract: Bi-temporal remote-sensing disaster change captioning often needs to identify sparse and spatially localized changes across large pre- and post-even

SALSA-V: Shortcut-Augmented Long-form Synchronized Audio from Videos

arXiv:2510.02916v2 Announce Type: replace-cross Abstract: We propose SALSA-V, a multimodal video-to-audio generation model capable of synthesizing highly synchronized, high-fidelity long-form audio fr

safetyarxiv-cs-lg

CASA: Content-Acoustic Speaking Assessment with Speech Encoder and Large Language Model

arXiv:2608.13101v1 Announce Type: new Abstract: Research on automatic speaking assessment (ASA) has increasingly adopted multimodal speech large language models to assess learners' speaking performanc

researcharxiv-cs-cl

EgoMonth: A Month-Level Egocentric Video Benchmark for Long-Term Spatiotemporal Memory

arXiv:2608.13113v1 Announce Type: cross Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have led to substantial progress in video understanding, accompanied by a growing number o

Exploring Instruction Data Quality for Explainable Image Quality Assessment

arXiv:2510.03880v2 Announce Type: replace Abstract: In recent years, with the rapid development of large multimodal models (LMMs), explainable image quality assessment (IQA) has attracted increasing a

Reasoning for Social Audio-Visual Question Answering: Where Do We Stand?

arXiv:2608.13239v1 Announce Type: new Abstract: Training Multimodal Large Language Models for audio-visual social understanding is a crucial step toward embodied social intelligence. Chain-of-thought

Spatial Memory Agent: Experience-Grounded Procedure Memory for Spatial Intelligence

arXiv:2608.12743v1 Announce Type: new Abstract: Spatial intelligence is becoming a foundation for embodied agents, robotic planning, and multimodal assistants. To improve the spatial reasoning ability

Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System

arXiv:2608.11738v1 Announce Type: cross Abstract: Multimodal Large Language Model (MLLM)-based UAV aerial image understanding and reasoning is essential for aerial intelligence yet poses distinct chal

Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization

arXiv:2608.12158v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have made rapid progress, yet they still exhibit object hallucination, generating plausible but incorrect descr

Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams

arXiv:2608.12262v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have been growing the capability for scientific writing and collaboration. For example, OpenAI Prism is a fre

ForgeryVCR: Visual-Centric Reasoning via Efficient Forensic Tools in MLLMs for Image Forgery Detection and Localization

arXiv:2602.14098v2 Announce Type: replace Abstract: Existing Multimodal Large Language Models (MLLMs) for image forgery detection and localization predominantly operate under a text-centric Chain-of-T

local-aiarxiv-cs-cv

GeoBridge: Decoupled Semantic Conditioning for Generative Image Geolocalization

arXiv:2608.11838v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have advanced image geolocalization mainly by improving how they reason about geographic cues. How that reasoni

GUIDE: Governed Unified Intelligence for Document-to-Artifact Generation in Enterprise Settings

arXiv:2608.12133v1 Announce Type: new Abstract: Enterprise guideline documents are heterogeneous and multimodal, combining narrative text, complex tables, and embedded images. Existing LLM and VLM sys

agentsarxiv-cs-ai

Self-Evolving Code-with-Image Reasoning

arXiv:2608.11292v1 Announce Type: new Abstract: Multimodal models increasingly reach for tools when solving visual tasks (crop, zoom, rotate, brighten), a paradigm known as thinking-with-images. The c

JEPA-WAM: Stage-Level Joint-Embedding Prediction for World-Action Models in Robot Manipulation

arXiv:2608.10780v1 Announce Type: new Abstract: Generalist robot policies aim to map multimodal observations and linguistic task instructions to actions across diverse tasks. However, existing methods

local-aiarxiv-cs-ro

12 Aug 2026

Order Matters: LVLMs as Judges for Temporal Reasoning in Image Sequences

arXiv:2608.10908v1 Announce Type: cross Abstract: As generative multimedia evolves from static image synthesis to complex, interleaved visual narratives, a foundational bottleneck has emerged: the jud

researcharxiv-cs-cl

12 Aug 2026

SafeCap: Improving LVLM Safety with Image Captioning Reinforcement Learning

arXiv:2608.10513v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) remain vulnerable to jailbreak attacks that exploit visual inputs to bypass safety alignment inherited from their

safetyarxiv-cs-ai

12 Aug 2026

ADOPD: Reference-Privileged On-Policy Distillation for MLLM-Based Industrial Anomaly Detection

arXiv:2608.09789v1 Announce Type: new Abstract: Industrial anomaly detection (IAD) requires identifying fine-grained deviations from normal visual patterns. Multimodal large language models (MLLMs) ca

Advantage-Guided Gate: Reshaping Open-Ended Reasoning for Vision-Based Spatial Intelligence

arXiv:2608.07987v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have demonstrated significant potential in complex spatial scene understanding and reasoning tasks. However, th

Beyond Global Editing: Per-Instance Disentangled Subspaces for Training-Free Hallucination Mitigation in LVLMs

arXiv:2608.09344v1 Announce Type: new Abstract: Recent advances in large vision-language models (LVLMs) have enabled powerful multimodal reasoning by integrating visual encoders with large language mo

Tutorials

Did the Grid Erase the Event? EndoClock for Auditing Medical World-Model Pipelines

arXiv:2608.09266v1 Announce Type: new Abstract: Medical world models commonly learn from multimodal recordings synchronized onto a fixed-rate grid. This preprocessing resamples each native stream onto

tutorialsarxiv-cs-cv

Findings of the First Teaching Monster Challenge: A Benchmark of Pedagogical Content Knowledge in AI Agents

arXiv:2608.08852v1 Announce Type: new Abstract: AI agents can now solve problems, answer like subject experts, and generate long-form multimodal content. However, whether they can adapt a lesson to fi

Hallucination-Free GUI Grounding via Regression-Free Layout-Aware Matching

arXiv:2608.09654v1 Announce Type: new Abstract: GUI agents are shifting from metadata-dependent large language models to purely visual multimodal large language models (MLLMs) that operate directly on

local-aiarxiv-cs-ai

Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models

arXiv:2608.09227v1 Announce Type: new Abstract: Omnimodal language models (OLMs) enable unified audio-visual understanding, but processing long joint token sequences makes inference computationally pr

OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories

model-releasesarxiv-cs-cl

arXiv:2608.08557v1 Announce Type: new Abstract: Visual tool use has emerged as a fundamental capability for multimodal agents to actively acquire evidence beyond a fixed image encoding. The prevailing

Performance of large language models in the optical diagnosis of colorectal polyps

arXiv:2608.07543v1 Announce Type: cross Abstract: Background and Study Aims: Accurate optical diagnosis of colorectal polyps guides resection strategy and surveillance, with multimodal large language

Preference Redirection via Attention Concentration: An Attack on Computer Use Agents

arXiv:2604.08005v2 Announce Type: replace Abstract: Advancements in multimodal foundation models have enabled the development of Computer Use Agents (CUAs) capable of autonomously interacting with GUI

agentsarxiv-cs-lg

Right Answer, Wrong Heat: Explanation-Aware Evaluation and Thermal-Grounded Feedback for MLLMs on Infrared Images

arXiv:2608.09145v1 Announce Type: new Abstract: General-purpose multimodal large language models (MLLMs) are increasingly applied to infrared images, where they are commonly scored by answer accuracy

Self-Evolving Neuro-Symbolic Skills for Tool-Augmented Spatial Reasoning

arXiv:2608.07955v1 Announce Type: new Abstract: Large vision-language models have achieved strong performance in multimodal reasoning, but they remain unreliable on fine-grained spatial tasks that dem

agentsarxiv-cs-ai

SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation

model-releasesarxiv-cs-ro

arXiv:2608.09771v1 Announce Type: new Abstract: Vision-language-action policies rely on large multimodal backbones to jointly perform perception, language conditioning, and action generation at every

The Judge Knows When It Knows: Calibrated Abstention for LLM-Based A/B-Test Prediction

model-releasesarxiv-cs-cl

arXiv:2608.07517v1 Announce Type: cross Abstract: Can a multimodal LLM predict which version of a web page will win a real A/B test from screenshots alone? We report the most complete answer we are aw

ToolVision: Learning When and How to Use Visual Tools with Capability-Aligned Supervision

arXiv:2608.08907v1 Announce Type: cross Abstract: Thinking with images allows a multimodal model to compensate for limited perception by invoking visual tools through code. Yet the prevailing SFT-then

agentsarxiv-cs-ai

VCU-Bridge: Hierarchical Visual Connotation Understanding via Semantic Bridging

arXiv:2511.18121v2 Announce Type: replace-cross Abstract: While Multimodal Large Language Models (MLLMs) excel on benchmarks, their processing paradigm differs from the human ability to integrate visu

Your VLM Already Knows When: Training-Free Temporal Grounding by Asking Yes or No

arXiv:2608.08315v1 Announce Type: new Abstract: Multimodal LLMs that recognise events reliably still fail to say when they happen. Prompted for timestamps, strong VLMs reach as little as 3.8% R@0.5 on