Search: “multimodal”

Information-Regularized Attention for Visual-Centric Reasoning

arXiv:2607.00434v1 Announce Type: new Abstract: Vision-language models (VLMs) have become a paradigm for multimodal learning, yet remain unstable due to object hallucination, weak visual grounding, an

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

Learning When to Listen: Gated Affect Fusion for Human Motion Prediction

DGX agent

arXiv:2607.00296v1 Announce Type: cross Abstract: Human motion forecasting in unconstrained real-world videos remains challenging due to the ambiguity of future behaviors and the presence of noisy mul

applicationsarxiv-cs-ai

2 Jul 2026

Linguistic Relative Policy Optimization for Video Anomaly Reasoning

arXiv:2607.00654v1 Announce Type: new Abstract: Video anomaly detection (VAD) with multimodal large language models has shown strong potential, yet most existing methods still depend on large-scale an

MindAU: EEG-Conditioned Facial Action Unit Editing via Dual-Stream Manifold Alignment

arXiv:2607.00410v1 Announce Type: new Abstract: Recent brain decoding studies have made substantial progress in reconstructing externally perceived visual content from neural signals. However, using e

RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarios

arXiv:2511.18011v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have demonstrated powerful capabilities in general spatial understanding and reasoning. However, their fine

StochasT: Learning with Stochastic Turn Depth for Visual Instruction Tuning

model-releasesarxiv-cs-cl

arXiv:2607.00465v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) rely extensively on Visual Instruction Tuning (VIT) to elicit their multimodal reasoning capabilities. However, w

Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptions

arXiv:2507.15692v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) provide new opportunities for blind and low vision (BLV) people to access visual information in their daily l

safetyarxiv-cs-cl

Synergistic Perception-Reasoning Governance: Grounding Medical MLLMs with Verifiable Anatomical Evidence

arXiv:2607.00060v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) show strong promise for clinical VQA and radiology report generation, yet inference-time hallucinations still u

Towards High-Resolution Visual Perception via Hierarchical Entity Exploration

arXiv:2607.00816v1 Announce Type: new Abstract: High-resolution (HR) image perception remains a key challenge in multimodal large language models (MLLMs), as fine-grained details are often lost when t

AMALIA-VL: A Native European Portuguese Open-Source Vision and Language Model

arXiv:2606.19100v2 Announce Type: replace Abstract: Large Vision and Language Models (LVLMs) have advanced rapidly, yet European Portuguese (pt-PT) remains systematically underserved by existing open-

DynFly: Dynamic-Aware Continuous Trajectory Generation for UAV Vision-Language Navigation in Urban Environments

arXiv:2606.31654v1 Announce Type: cross Abstract: Recent advances in multimodal large models have significantly improved UAV vision-language navigation (UAV-VLN) by enhancing high-level perception and

EgoVITA: Learning to Plan and Verify for Egocentric Video Reasoning

arXiv:2511.18242v3 Announce Type: replace Abstract: Egocentric video understanding requires procedural reasoning under partial observability and continuously shifting viewpoints. Current multimodal la

Local Ai

ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education

arXiv:2606.30662v1 Announce Type: cross Abstract: The advent of Generative Artificial Intelligence (GenAI), and in particular Large Language Models (LLMs), is reshaping educational practice, while int

local-aiarxiv-cs-ai

ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs

arXiv:2606.31982v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) incur prohibitive inference costs due to long visual token sequences. Training-free visual token reduction prov

Evo-PI: Aligning Medical Reasoning via Evolving Principle-Guided Supervision

arXiv:2606.31800v1 Announce Type: new Abstract: Despite recent progress, the reasoning capabilities of large multimodal language models (MLLMs) remain fundamentally constrained by static supervision,

safetyarxiv-cs-ai

How Post-Training Shapes Biological Reasoning Models

arXiv:2606.16517v2 Announce Type: replace Abstract: Scientific reasoning models for biology combine language models with foundation models trained on multimodal biological data, including DNA, RNA, an

researcharxiv-cs-lg

Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents

arXiv:2606.31270v1 Announce Type: cross Abstract: Computer-use agents, which leverage multimodal large language models (MLLMs) to operate computers and complete tasks, have attracted significant atten

ReGRPO: Reflection-Augmented Policy Optimization for Tool-Using Agents

arXiv:2606.31392v1 Announce Type: new Abstract: Tool-augmented vision-language models (VLMs) can solve multimodal, multi-step tasks by calling external tools, yet they remain fragile in practice. Exis

safetyarxiv-cs-ai

Spatial Reasoning via Modality Switching Between Language and Symbolic Representation

arXiv:2606.31285v1 Announce Type: new Abstract: Human reasoning is inherently multimodal: when problems become difficult, we rarely think in words alone. We often externalize our reasoning by sketchin

A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosis

arXiv:2606.29193v1 Announce Type: cross Abstract: LLM-based agents are reshaping microservice operations into AgentOps, where benchmarks are key to evaluating failure diagnosis over multimodal observa

Agents

Agents-K1: Towards Agent-native Knowledge Orchestration

arXiv:2606.13669v2 Announce Type: replace Abstract: Current LLM-based research agents have advanced through agent orchestration, yet largely overlook scientific knowledge orchestration. Existing works

agentsarxiv-cs-ai

Can OCR-VLMs Read Devanagari? A Stress-Test Benchmark and Post-Correction Study

model-releasesarxiv-cs-cl

arXiv:2606.29213v1 Announce Type: new Abstract: OCR systems, ranging from classical engines to specialised OCR vision-language models (OCR-VLMs) and frontier multimodal LLMs, report strong results on

DLR: Zero-Inference-Cost Latent Residuals for Low-Rank Pre-Training

arXiv:2606.28932v1 Announce Type: cross Abstract: Large language models have driven recent progress in language and multimodal AI, yet pre-training them at scale is prohibitively expensive. Low-rank p

Does Role Specialization Matter for Explanation Faithfulness in Mixture-of-Experts?

arXiv:2606.29613v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) architectures have recently been extended with role-based mechanisms for interpretability. This is typically done by assignin

Agents

Efficient-VLN: A Simple yet Strong Baseline for Efficient Vision-Language Navigation

arXiv:2512.10310v2 Announce Type: replace Abstract: While Multimodal Large Language Models (MLLMs) have demonstrated significant promise in Vision-Language Navigation (VLN), existing agents remain hea

agentsarxiv-cs-cv

EfficientUICoder: A Bidirectional Token Compression Framework for Efficient MLLM-Based UI Code Generation

arXiv:2509.12159v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models have demonstrated exceptional performance in UI2Code tasks, significantly enhancing website development effic

Enhancing Part-Level Point Grounding for Any Open-Source MLLMs

arXiv:2606.29267v1 Announce Type: new Abstract: Visual grounding aims to associate free-form textual queries with specific regions in an image. While recent Multimodal Large Language Models (MLLMs) ha

Exploiting Vision Encoder Vulnerabilities for Universal Adversarial Perturbations on Large Vision-Language Models

arXiv:2412.08108v3 Announce Type: replace-cross Abstract: Large Vision-Language Models (LVLMs) have achieved remarkable performance on multimodal tasks but remain highly vulnerable to small adversaria

researcharxiv-cs-cl

Tutorials

Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding

arXiv:2603.19235v2 Announce Type: replace Abstract: While Multimodal Large Language Models demonstrate impressive semantic capabilities, they often suffer from spatial blindness, struggling with fine-

tutorialsarxiv-cs-cv

How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning

model-releasesarxiv-cs-cl

arXiv:2606.29672v1 Announce Type: new Abstract: Evaluating the originality of visual images poses enduring challenges for creativity assessment. Automated scoring using AI models has proven effective

HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding

arXiv:2602.12957v3 Announce Type: replace Abstract: Document parsing is a fundamental task in multimodal understanding, supporting a wide range of downstream applications such as information extractio

Applications

Latent-CURE for Breast Cancer Diagnosis

arXiv:2606.29928v1 Announce Type: cross Abstract: Multimodal Large Models have significantly advanced automated breast ultrasound diagnosis. However, most existing frameworks utilize opaque, end-to-en

applicationsarxiv-cs-ai

Learning from Reliable Latent Prompts for Visual Recognition with Missing Modalities

arXiv:2606.30597v1 Announce Type: new Abstract: Large-scale multimodal models (LMMs) have achieved superior performance in visual recognition by synergizing information across diverse, massive-scale p

MIRROR: Aligning Semantic Relations from Language to Image via Gromov--Wasserstein

arXiv:2606.29462v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) inherit rich relational priors from their language backbones, yet often fail when asked to apply these relation

OmniCoT: A Benchmark for Global and Multi-Step Panoramic Reasoning

arXiv:2606.30378v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated promising spatial reasoning capabilities, while these abilities remain underexplored in the e

Primary ICD Category Prediction using LLM-based Probing

arXiv:2606.28798v1 Announce Type: new Abstract: Objective: ICD codes are central to reimbursement, research, and population health surveillance, yet automated coding systems often struggle to integrat

Rigel: Self-Distilled Score Adaptation for Image and Video Captioning Evaluation

arXiv:2606.29997v1 Announce Type: new Abstract: Automatic evaluation of image and video captioning is essential for benchmarking multimodal systems, although standard evaluation metrics show limited a

Local Ai

See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs

arXiv:2606.29847v1 Announce Type: new Abstract: Large Vision-Language Models (LVLMs) excel at multimodal tasks but remain prone to object hallucinations. Prior training-free remedies often uniformly s

local-aiarxiv-cs-cv

Set-Inclusive Uncertainty Modeling for Robust Brain Tumor Segmentation

arXiv:2606.30374v1 Announce Type: cross Abstract: Multimodal MRI is essential for accurate brain tumor segmentation. However, acquiring all modalities at inference is often challenging in practice, wh

VIGIL: Part-Grounded Structured Reasoning for Generalizable Deepfake Detection

arXiv:2603.21526v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) offer a promising path toward interpretable deepfake detection by generating textual explanations. However,

EXPLORE-Bench: Egocentric Scene Prediction with Long-Horizon Reasoning

arXiv:2603.09731v3 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) are increasingly considered as a foundation for embodied agents, yet it remains unclear whether they

HumanMoveVQA: Can Video MLLMs reason about human movement in videos?

arXiv:2606.27999v1 Announce Type: new Abstract: Despite the rapid advance of Multimodal Large Language Models (MLLMs) in high-level video understanding, a fundamental bottleneck remains: these models

HybridCodec: Modeling Discrete and Continuous Representations for Efficient Speech Language Models

arXiv:2606.27627v1 Announce Type: cross Abstract: Discrete audio representations have become increasingly popular for building multimodal text-audio systems and integrating audio capabilities into Lar

NormAct: A Benchmark for Hidden Social Norm Compliance in Embodied Planning

arXiv:2606.27826v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly deployed as embodied planners in egocentric environments, where task success requires not only

Reflect-R1: Evidence-Driven Reflection for Self-Correction in Long Video Understanding

arXiv:2606.27922v1 Announce Type: cross Abstract: Current multimodal reflection mechanisms for long video understanding predominantly rely on closed-loop self-reflection within internal parameters. La

safetyarxiv-cs-ai

Understanding How MLLMs Describe Artworks Using Token Activation Maps

arXiv:2606.27947v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) describe artworks with remarkable fluency, yet the visual reasoning behind their outputs remains opaque. When a

Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoning

arXiv:2606.27828v1 Announce Type: new Abstract: Recent interest in multimodal large language models (MLLMs) raises a central question: can they reason over dynamic visual evidence rather than merely r

CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs

arXiv:2606.27264v1 Announce Type: new Abstract: Reasoning in multimodal large language models (MLLMs) has shown strong promise in medical imaging. However, this reasoning is usually free-form text jud