Search: “multimodal”

Scene Graph-Driven Haptic Feedback for Safety Enhancement in Robotic Ophthalmic Surgery via Physically Simulated iOCT

arXiv:2609.07857v1 Announce Type: cross Abstract: Robotic ophthalmic surgery offers high precision but introduces a 'sensory gap' by decoupling the surgeon from their instrument, resulting in a loss o

safetyarxiv-cs-cv

9 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

MedQA-MM: Shortcuts Behind Medical Visual Reasoning

DGX agent

arXiv:2609.03261v1 Announce Type: cross Abstract: A benchmark score credits final answers, but not the route by which an item can be answered. In medical multimodal multiple-choice questions (MCQs), t

model-releasesarxiv-cs-cl

4 Sept 2026

MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT

arXiv:2609.03690v1 Announce Type: new Abstract: The joint interpretation of metabolic function and anatomical structure is essential for clinical diagnosis in whole-body PET/CT. Although recent advanc

4 Sept 2026

Urban Boundaries, Social Barriers: A Benchmark and Vision-Centric Framework for Mapping Gated Communities and Equity Implications

arXiv:2609.03804v1 Announce Type: new Abstract: Communities are fundamental spatial units that shape urban form and social life. Whether a residential compound is spatially open or enclosed affects mo

4 Sept 2026

DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents

arXiv:2609.02059v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved strong performance on structured visual understanding tasks such as chart and document question a

DynaTokens: Controlling Token Dynamics for Continual Video-Language Understanding

arXiv:2603.06662v3 Announce Type: replace Abstract: Continual VideoQA with multimodal LLMs remains challenging because sequential adaptation induces task interference, while storing task-specific prom

InfraPatch: Cross-Task Targeted Grayscale Patch Attacks on Infrared-Adapted Vision-Language Models

arXiv:2609.02233v1 Announce Type: cross Abstract: Infrared vision-language models (IR-VLMs) have emerged as a promising paradigm for multimodal perception under low-visibility conditions, yet their ro

local-aiarxiv-cs-ai

Video Object Segmentation-Aware Audio Generation

arXiv:2509.26604v2 Announce Type: replace Abstract: Existing multimodal audio generation models often lack precise user control, which limits their applicability in professional Foley workflows. In pa

ClearText-Video: A Large-Scale Text-Centric Video Dataset Bridging Video Restoration and Scene-Text Enhancement

arXiv:2608.28784v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have recently made strong progress in visual--linguistic understanding. However, their performance on text-cent

Co-Evolutionary Prompt Optimization with Cross-Category Transfer for Zero-Shot Anomaly Detection

arXiv:2608.29467v1 Announce Type: new Abstract: Zero-shot anomaly detection (ZSAD) has gained significant attention for its practical value in industrial inspection. Recently, CLIP-based approaches ha

Defending Wearable VLMs Against Private Attribute Inference

arXiv:2608.28691v1 Announce Type: cross Abstract: Wearable VLM pipelines promise continuous multimodal assistance from egocentric visual capture: a user asks a task-driven question about the surroundi

Evidence-Bounded Mental Health Reasoning from Heterogeneous Speech Protocols

model-releasesarxiv-cs-cl

arXiv:2608.31014v1 Announce Type: new Abstract: Computational mental health screening using multimodal speech and text has shown great promise. However, existing models often assume all clinical speec

Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Models

arXiv:2608.29590v1 Announce Type: new Abstract: We propose a societal bias evaluation method for large vision-language models (LVLMs) in the era of strong safety guardrails. Existing benchmarks rely o

RAGDiffusion++: From Macro-Retrieval to Micro-Fidelity Alignment for Garment Generation

arXiv:2608.29280v1 Announce Type: cross Abstract: Standard clothing asset generation---restoring forward-facing flat-lay garment images from diverse real-world contexts---holds immense commercial valu

Subtraction-Based Tumor Segmentation and Lesion-Centered pCR Prediction for the MAMA-MIA Challenge

arXiv:2608.29162v1 Announce Type: cross Abstract: We describe the submission of team FME to the MAMA-MIA Challenge, which evaluated primary tumor segmentation and prediction of pathological complete r

TopoAgent: A Structure-Aware Perception-to-Reasoning Framework for Diagram-to-Graph Topology Extraction with Large Vision-Language Models

arXiv:2608.28701v1 Announce Type: new Abstract: Diagram-to-graph topology extraction aims to extract a graph of entities and their connections from a structural diagram. This task remains challenging

XQDT: eXplainable and Quantitative Data-Text Alignment Metric with Feedback Signals

arXiv:2608.29948v1 Announce Type: new Abstract: Evaluating data-text alignment remains challenging: existing metrics often provide limited explanations for the scores, while prompt-based LLM-as-Judge

local-aiarxiv-cs-cl

NL2AGBench: Benchmarking LLM Auto-Formalization for AlphaGeometry

model-releasesarxiv-cs-cl

arXiv:2608.28481v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have demonstrated strong capabilities in natural language understanding and mathematical reasoning. Howe

31 Aug 2026

AesCanvas: A Large-Scale Dataset and Benchmark for Aesthetic Critique and Contextual Suitability

arXiv:2608.26713v1 Announce Type: cross Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have extended Image Aesthetic Assessment (IAA) beyond scalar scores toward interpretable c

AffectOmni: RL-Verifiable People-Centric Grounded Affective Reasoning for Social and Art-Related Scenes

arXiv:2608.26193v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) achieve strong performance on VQA and scene understanding, yet affective reasoning remains vulnerable to shortc

researcharxiv-cs-ai

Ancient-Bench: A Comprehensive Multi-millennial, Multi-medium, and Multi-script Benchmark for Ancient Chinese Artifact Text Recognition

arXiv:2608.27169v1 Announce Type: new Abstract: Ancient Chinese artifact text recognition is fundamental to heritage digitization, and benchmarks for ancient texts are essential for evaluating current

Applications

DOD-SA: Infrared-Visible Decoupled Object Detection with Single-Modality Annotations

arXiv:2508.10445v2 Announce Type: replace Abstract: Infrared-visible object detection has shown great potential in real-world applications, enabling robust all-day perception by leveraging the complem

applicationsarxiv-cs-cv

Information-Guided Frontier Decoding: Contextual Utility-Driven Commitment in dMLLMs

arXiv:2608.26641v1 Announce Type: new Abstract: Decoding quality in diffusion multimodal language models (dMLLMs) depends heavily on the order in which masked tokens are committed. Existing confidence

local-aiarxiv-cs-cl

Multivariate Diffusion Transformer with Decoupled Attention for High-Fidelity Mask-Text Collaborative Facial Generation

arXiv:2511.12631v3 Announce Type: replace-cross Abstract: While significant progress has been achieved in multimodal facial generation using semantic masks and textual descriptions, conventional featu

researcharxiv-cs-ai

Order Matters: A Chinese Multi-Panel Meme Benchmark for Vision-Language Reasoning

arXiv:2608.26866v1 Announce Type: new Abstract: Many multimodal tasks depend on how visual elements are ordered and composed, not only on recognizing them in isolation. Internet memes are a compact ca

UrbanGround: From Local Perception to Spatial Agency in a Real-Scale City

arXiv:2608.27456v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can interpret a street view, but urban agency depends on whether such local evidence remains useful after the a

local-aiarxiv-cs-cv

VIPER: An Expert-Curated Benchmark for Vision-Language Models in Veterinary Pathology

arXiv:2608.26382v1 Announce Type: new Abstract: Pathology vision-language models are advancing rapidly, yet existing benchmarks remain focused on human tissue, particularly oncology, leaving non-human

ARGenSeg: Image Segmentation with Autoregressive Image Generation Model

arXiv:2510.20803v2 Announce Type: replace Abstract: We propose a novel AutoRegressive Generation-based paradigm for image Segmentation (ARGenSeg), achieving multimodal understanding and pixel-level pe

CVE-SAI: Counterfactual Visual Evidence-Guided Selective Attribute Indexing for Risk-Controlled E-commerce Search

arXiv:2608.25023v1 Announce Type: cross Abstract: Multimodal product models can complete missing e-commerce attributes, yet current methods still optimize attribute-answer accuracy without verifying v

local-aiarxiv-cs-cv

Modality Contribution Score - A Per-Patient Framework for Quantifying the Relative Diagnostic Contribution of Structural MRI and Amyloid PET in Alzheimer's Disease

arXiv:2608.24931v1 Announce Type: cross Abstract: Multimodal neuroimaging combining structural MRI and positron emission tomography (PET) captures complementary structure-function relationships across

MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching

arXiv:2608.26094v1 Announce Type: new Abstract: Existing action quality assessment (AQA) datasets and methods rely primarily on visual inputs such as RGB and pose, overlooking physiological dynamics s

Resolving Multi-Modal Regression by Difference-Quotient-Based Clustering:Fast Coarse Conditional-Label Assignment

arXiv:2608.25467v1 Announce Type: new Abstract: Multimodal regression suffers from the mean-collapse pathology: under squared loss, an unconstrained regressor converges to the conditional mean, which

researcharxiv-cs-lg

What Do Medical Vision-Language Models Learn in Radiology? Transfer, Alignment, and Source-Proxy Leakage Under Distribution Shift

arXiv:2608.25251v1 Announce Type: new Abstract: Medical vision-language models (VLMs) can appear reliable in-domain while failing when acquisition domain, paired supervision, or evaluation protocol ch

safetyarxiv-cs-cv

Differential Learning for Robust Prediction of Thermal Stability with Application to Energetic Materials

arXiv:2608.23874v1 Announce Type: cross Abstract: Predicting thermal stability during handling and storage is essential for the design of safe and reliable energetic materials. However, experimental m

safetyarxiv-cs-lg

EXAM^2: nderline{Ex}tending nderline{A}udio Understanding in nderline{M}ultilingual and nderline{M}ultimodal Analysis

arXiv:2608.23758v1 Announce Type: cross Abstract: Recent large audio language models (LALMs) have achieved impressive progress in audio understanding. However, existing evaluations remain largely cons

PhysMLLMs: Spatial Priors for Unified Referring Segmentation and Grounded Reasoning of Images and Videos

arXiv:2608.24574v1 Announce Type: new Abstract: Video multimodal large language models support language guided video segmentation, but they often show spatio temporal inconsistencies, e.g., jitter, dr

Applications

When Seeing Is Not Enough: Benchmarking Interactive Visual Grounding in LVLMs

arXiv:2608.23978v1 Announce Type: new Abstract: Visual grounding is typically evaluated as a one-shot mapping from an informative referring expression to a visual target. This formulation misses a cen

applicationsarxiv-cs-ai

Applications

Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models

arXiv:2608.23478v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models can turn multimodal context into robot actions, but their action decoders are still trained largely by behavior cl

applicationsarxiv-cs-ai

Budget-Constrained Embodied Perception: Four Resource Walls and a Pre-Registered Evaluation of Access-Structured Perception on Open Models at less than 31B

arXiv:2608.22975v1 Announce Type: new Abstract: Embodied multimodal agents must answer from growing observation streams under a fixed per-decision token budget. We formalize this constraint through fo

Buried in Textual Debt: Context Pruning with Visual Evidence Preservation for MLLM Agents

arXiv:2608.22963v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed as multi-step agents, where explicit reasoning supports task decomposition and tool c

HeatTok: Enhancing Remote Sensing Image Understanding via Thermodiffusion-based Tokenization

arXiv:2608.22485v1 Announce Type: new Abstract: Current visual tokenizers in Multimodal Large Language Models (MLLMs) predominantly rely on patch-based partitioning, which causes severe semantic mixtu

MLLM-Assisted Audio VOS: A 3rd Place Report for the MeViS-Audio Track, 8th LSVOS Challenge

arXiv:2608.23234v1 Announce Type: new Abstract: In this technical report, we present a training-free framework for audio-guided video object segmentation, which integrates Multimodal Large Language Mo

Do SpeechLMs Hear Their Own Opinions? Diagnosing and Mitigating Previous-Belief Contamination in Streaming Emotion Understanding

arXiv:2608.20769v1 Announce Type: cross Abstract: Streaming emotion understanding uses historical state while continuously interpreting current audio, often feeding the model's previous prediction bac

24 Aug 2026

Identify, Locate, Link: End-to-End Key-Value Extraction from Document Images

model-releasesarxiv-cs-cl

arXiv:2608.20868v1 Announce Type: cross Abstract: Document processing pipelines traditionally cascade optical character recognition (OCR) engines with downstream models for structured information extr

24 Aug 2026

Agents

LingShu: A Large-Scale Symptom-Centric Contextualized Knowledge Graph Bridging Traditional Chinese Medicine and Modern Biomedicine

arXiv:2608.20402v1 Announce Type: cross Abstract: Biomedical knowledge graphs (KGs) are pivotal for knowledge organization, yet traditional binary relations often struggle to represent the conditional

agentsarxiv-cs-ai

24 Aug 2026

Extended to Reality: Prompt Injection in 3D Environments

arXiv:2602.07104v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have advanced the capabilities to interpret and act on visual input in 3D environments, empowering di

researcharxiv-cs-ai

21 Aug 2026

Projector Is All You Train

arXiv:2608.19726v1 Announce Type: new Abstract: The typical training process of a multimodal large language model (MLLM) involves adapting both the language model backbone and the projector between th

researcharxiv-cs-cl

21 Aug 2026

Tutorials

Role-Conditioned Sub-Token Routing for Efficient Vision-Language-Action Policies