Search: “multimodal”

arXiv:2503.09197v2 Announce Type: replace Abstract: Image quality scoring and interpreting are two fundamental components of Image Quality Assessment (IQA). The former quantifies image quality, while

tutorialsarxiv-cs-cv

15 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

UniCAR-RL: Seeing Better before Thinking Deeper in Visual Mathematics

DGX agent

arXiv:2609.13849v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) often struggle with complex mathematical visual reasoning primarily due to a lack of fine-grained perception, c

safetyarxiv-cs-ai

15 Sept 2026

VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understanding

arXiv:2609.15606v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress on short video understanding yet remain limited on long videos due to the l

15 Sept 2026

Applications

A Dual Cross-Attention Framework for Colposcopic CIN Grading and Swede Score Prediction Using a New Multi-Center Dataset

arXiv:2609.12827v1 Announce Type: new Abstract: Cervical cancer is a major global health challenge, with disease burden falling disproportionately on low- and middle-income countries (LMICs) due to a

applicationsarxiv-cs-cv

DropVLA: An Action-Level Backdoor Attack on Vision-Language-Action Models

arXiv:2510.10932v5 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models map multimodal perception and language instructions to executable robot actions, making them particularly

safetyarxiv-cs-ro

Impact of Multiple Non-Invasive Biosignals on Cardiovascular Biomarker Estimation via Simulation-Based Inference

arXiv:2609.11969v1 Announce Type: cross Abstract: As the population ages, the number of patients with cardiovascular diseases continues to increase, highlighting the need for early detection before pr

researcharxiv-cs-lg

Observation-Anchored Selective Assimilation for Longitudinal Tumor-State Proxy Forecasting in Post-Treatment Glioma

arXiv:2609.12435v1 Announce Type: new Abstract: Post-treatment MRI in patients with glioma provides serial observations for updating patient-specific tumor-state proxy estimates, but variable appearan

researcharxiv-cs-lg

ProactiveBench: Can Streaming Video Models Really Interact Like Humans?

arXiv:2609.12658v1 Announce Type: new Abstract: Streaming video understanding requires models to process continuous multimodal input while maintaining temporal context. Existing evaluations are predom

researcharxiv-cs-lg

Same Encoder, Different Winner: A Paired-View Framework for Cell Painting Encoder Evaluation

arXiv:2609.12761v1 Announce Type: new Abstract: Vision encoders for Cell Painting are typically ranked by a single evaluation, commonly replicate mean average precision (mAP). We introduce CP-BG-Bench

What Did the MLLM Hear? Token-Level Spectro-Temporal Grounding for Audio MLLM Explainability

arXiv:2609.12663v1 Announce Type: cross Abstract: Audio-based Multimodal Large Language Models (MLLMs) can generate detailed natural-language descriptions of complex acoustic scenes, yet it remains un

IMLE-VLA: Fast Single-Step Action Generation for Vision-Language-Action Policies

arXiv:2609.10915v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies leverage pretrained vision-language backbones to achieve strong cross-task generalization. A leading design coup

11 Sept 2026

Applications

Towards AI-Driven Policing: Interdisciplinary Knowledge Discovery from Police Body-Worn Camera Footage

arXiv:2504.20007v4 Announce Type: replace-cross Abstract: This paper proposes a novel interdisciplinary framework for analyzing police body-worn camera (BWC) footage from the Rochester Police Departme

applicationsarxiv-cs-cv

11 Sept 2026

Beyond One-Size-Fits-All: Sample-Adaptive Strategy Routing for Vision Token Pruning in MLLMs

arXiv:2609.10346v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) process hundreds or thousands of visual tokens per image, incurring prohibitive inference costs. While existing

From Symbolic Perception to Logical Deduction: A Framework for Guiding Language Models in Geometric Reasoning

model-releasesarxiv-cs-cl

arXiv:2609.10335v1 Announce Type: cross Abstract: Plane geometry remains a significant challenge in AI, requiring the integration of visual perception and mathematical reasoning. While Large Multimoda

Isotropic Embedding Perturbations for Robust Vision Language Encoders

arXiv:2609.10292v1 Announce Type: new Abstract: Data augmentation is fundamental to training modern deep vision and multimodal models. While individual methods, such as RandAug, CutMix, Mixup, RandEra

safetyarxiv-cs-cv

MLLMs Hallucinate when Information Distribution Drifts in Synergy Heads

arXiv:2609.09206v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) often struggle with hallucinations, thus hindering their reliable practical applications. Existing attention-

PccDiffuser: Multi-solution Motion Planning for Continuum Robots

arXiv:2609.09745v1 Announce Type: new Abstract: We present the PccDiffuser, a conditional diffusion framework for continuum robots that learns a multimodal distribution over complete configuration-spa

researcharxiv-cs-ro

S^3-Bench: Evaluating Speech Interaction Models as Scientific Voice Assistants

arXiv:2609.09852v1 Announce Type: new Abstract: The advance of multimodal large language models (MLLMs) has fundamentally reshaped the paradigm of human-computer interaction, especially speech interac

SEA-SpeechBench: A Large-Scale Multitask Benchmark for Speech Understanding Across Southeast Asia

model-releasesarxiv-cs-cl

arXiv:2609.09672v1 Announce Type: new Abstract: The rapid advancement of audio and multimodal large language models has unlocked transformative speech understanding capabilities, yet evaluation framew

VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoning

arXiv:2603.25021v3 Announce Type: replace Abstract: Existing Multimodal Large Language Models (MLLMs) often suffer from hallucinations in long video understanding (LVU), primarily due to the imbalance

safetyarxiv-cs-cv

CAM: Question Answering on Entity-Centric Videos with Continuous Extraction and Adaptive Querying

arXiv:2609.06504v1 Announce Type: new Abstract: Memory facilitates question answering over long videos by extracting and retrieving facts to fit within the limited context windows of multimodal LLMs (

How Well Can Modern LLMs Act as Agent Cores in Radiology Environments?

arXiv:2412.09529v4 Announce Type: replace Abstract: Radiology, with its heterogeneous modalities, anatomies, and evolving protocols, is a natural yet high-stakes testbed for agentic AI. As LLMs grow m

LASSNet: Level-Aware Availability-Conditioned Spatial-Semantic Fusion for Brain Tumor Segmentation with Missing MRI Modalities

arXiv:2609.06733v1 Announce Type: new Abstract: Brain tumor segmentation from multimodal MRI relies on complementary evidence across four imaging sequences, yet one or more modalities may be unavailab

NutriBench-Kitchen: Benchmarking Embodied AI for Nutrition Management

arXiv:2609.07135v1 Announce Type: new Abstract: An embodied kitchen assistant must do more than recognize food in isolated frames. It must track ingredient states over time and integrate visual observ

STAR-Pro: Stage-Wise Token Adaptive Reduction with Progressive Refinement for Efficient Large Vision-Language Models

arXiv:2609.05916v1 Announce Type: new Abstract: Large vision-language models (LVLMs) achieve strong multimodal understanding, but the hundreds to thousands of visual tokens they process impose substan

TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs

arXiv:2602.00288v4 Announce Type: replace Abstract: Fine-grained spatio-temporal understanding is essential for video reasoning and embodied AI. Yet, while Multimodal Large Language Models (MLLMs) mas

EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agents

arXiv:2607.09773v2 Announce Type: replace Abstract: Computer-use agents must solve long-horizon tasks through repeated interaction with partially observable, multimodal desktop environments. Although

FailSAE: Towards Interpretable Failure Prediction for Vision-Language Models via Sparse Autoencoders

arXiv:2609.04276v1 Announce Type: new Abstract: Vision-language models (VLMs), such as CLIP, have achieved strong performance across multimodal tasks by aligning visual and textual representations in

Agents

First Things First: Teaching LLM-Based Agents to Prioritize Must-Haves before Nice-to-Haves

arXiv:2609.05224v1 Announce Type: new Abstract: Recent progress in multimodal large language models (MLLMs) has fueled significant enthusiasm in their potential to act as autonomous agents for real-wo

agentsarxiv-cs-cv

MM-IFEval-Pro: A Multilingual and Attack-Resistant Benchmark for Instruction-Following in Vision-Language Models

arXiv:2609.04859v1 Announce Type: new Abstract: As vision-language models (VLMs) rapidly advance in image understanding, cross-modal reasoning, and complex instruction execution, instruction-following

When Seeing Overrides Knowing: Visual Dominance and Deferral-Based Method for Personalized Safety in VLMs

arXiv:2609.04281v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly deployed in high-stakes settings, where a response that is reasonable in general may still be unsafe fo

Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks

arXiv:2609.03734v1 Announce Type: cross Abstract: BLEU-4 is the standard metric for evaluating sign language translation (SLT), but spoken-language metrics may not adequately reflect sign language pro

researcharxiv-cs-ai

Imagination Helps Visual Reasoning, But Not Yet in Latent Space

arXiv:2602.22766v3 Announce Type: replace Abstract: Latent visual reasoning aims to mimic human's imagination process by meditating through hidden states of Multimodal Large Language Models. While rec

Sensing Which Modality Matters: Evidence-Gated Regularization for Robust VLA Policies

arXiv:2609.03142v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) policies fuse multimodal sensory inputs, but training on limited and homogeneous robot demonstrations encourages spurious

STAIR (STructure Aware Information Retriever): A novel dataset and LLM based retriever for document structure augmentation

arXiv:2609.03874v1 Announce Type: new Abstract: Retrieval Augmented Generation (RAG) is a key component for generating accurate and hallucination free answers using Large Language Models (LLMs). LLMs

Unfold The World: Factorize 4D Properties in Reinforcing Spatial Reasoning

arXiv:2609.03729v1 Announce Type: new Abstract: Despite the remarkable prowess of Vision-Language Models (VLMs) in general multimodal tasks, they remain fundamentally ``flat'' when reasoning about the

safetyarxiv-cs-cv

Beauty is in the AI of the beholder: MLLMs systematically overrate facial attractiveness

arXiv:2609.02512v1 Announce Type: new Abstract: Beauty assessments from Multimodal Large Language Models (MLLMs) are increasingly popular amongst users, companies, and aestheticians. This raises the q

DiDrive: A Risk-Aware Hierarchical Diffusion Framework for Safe Offline Reinforcement Learning in Autonomous Driving

model-releasesarxiv-cs-lg

arXiv:2609.01609v1 Announce Type: new Abstract: While diffusion models effectively capture multimodal behavioral priors for autonomous driving, offline reinforcement learning (RL) policies remain susc

Local Ai

Subcellularly Resolved Single-Cell Embedding Learning with Transcriptomic data, Protein Structure and Localization Information

arXiv:2609.02344v1 Announce Type: cross Abstract: Existing cell embedding methods predominantly rely on transcriptomic or proteomic measurements and represent each cell as a holistic entity, thereby o

local-aiarxiv-cs-ai

Agents

Towards Zero-Shot Transfer Across Embodiments For Driving VLAs

arXiv:2609.02341v1 Announce Type: new Abstract: Vision-Language-Action models (VLAs) have shown strong potential in autonomous driving by leveraging multimodal pretraining for instruction following, v

agentsarxiv-cs-cv

Video2Reaction: Training Foundation Video Models to Predict Audience Reaction

arXiv:2609.01816v1 Announce Type: new Abstract: We introduce Video2Reaction, a multimodal dataset that maps short movie segments to the induced emotional reactions of viewers in the wild, as expressed

Local Ai

A Glance Is All You Need: Single-Pass Fine-Grained Image Captioning with SimLoss

arXiv:2609.00591v1 Announce Type: new Abstract: An image may be worth a thousand words, but most captioning models describe it in only a few. Modern vision-language models produce fluent high-level ca

local-aiarxiv-cs-cv

ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues

model-releasesarxiv-cs-cl

arXiv:2609.01111v1 Announce Type: new Abstract: Clinical LLM assistants must reason over multi-visit patient trajectories, yet whether the compact history representations used to scale them---retrieva

ExBind: A Controlled Diagnostic Benchmark for Visual-to-Executable Correspondence

arXiv:2609.01344v1 Announce Type: new Abstract: Multimodal coding and editing systems must map a visible or semantic referent to the exact executable object that can be edited. A wrong reference may s

Jailbreaking Text-to-Image Models Through Cracks: Navigating Heterogeneous Safety Filters via Multi-Agent Debate

arXiv:2609.01168v1 Announce Type: new Abstract: Text-to-image (T2I) models remain vulnerable to jailbreak attacks that elicit Not-Safe-For-Work (NSFW) content, despite increasingly being guarded by he

safetyarxiv-cs-ai

Local Ai

Risk-Aware Decision-Making for Autonomous Overtaking: A World Model-Based Mixture-of-Experts Framework

arXiv:2609.00385v1 Announce Type: cross Abstract: Autonomous highway overtaking demands foresighted decision-making to handle complex interactions, stochastic traffic evolution, and temporal risk accu

local-aiarxiv-cs-ai

The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence

arXiv:2609.00868v1 Announce Type: cross Abstract: Vision-language models are evaluated by aggregate accuracy on multimodal benchmarks, a practice that implicitly assumes the model uses its visual inpu

UI-Venus-2 Technical Report

arXiv:2609.00028v1 Announce Type: new Abstract: Multimodal GUI agents have emerged as a promising paradigm for digital task automation, yet transitioning from benchmark-oriented models to dependable r