Search: “multimodal”

BabyVision: Visual Reasoning Beyond Language

arXiv:2601.06521v2 Announce Type: replace-cross Abstract: While humans develop core visual skills long before acquiring language, contemporary Multimodal LLMs (MLLMs) still rely heavily on linguistic

BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2607.05614v1 Announce Type: cross Abstract: Document comprehension is a challenging yet impactful task for Multimodal Large Language Models, especially as these systems see growing adoption in r

model-releasesarxiv-cs-ai

8 Jul 2026

Cross-Contextual Vision-Language Adaptation with LoRA for Personalized Severe Adverse Event Detection in Clinical Wound Monitoring

arXiv:2607.05625v1 Announce Type: new Abstract: Wound monitoring is a critical yet underserved clinical challenge, where timely identification of severe adverse events (SAEs) such as infection, tissue

Applications

Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator

arXiv:2607.05765v1 Announce Type: new Abstract: Embodied navigation aims to build agents that interpret multimodal goals, reason in 3D space, and reach target destinations reliably in the real world.

applicationsarxiv-cs-cv

Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis

arXiv:2602.00846v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) struggle with alignment due to the limitations of existing reward models (RMs), which are predominantly vis

Optimal Transport Q-Learning for Flow Policy Steering and Acceleration

arXiv:2607.06262v1 Announce Type: new Abstract: Diffusion and flow policies have recently demonstrated remarkable performance in robotic applications by accurately capturing multimodal robot trajector

safetyarxiv-cs-ro

Local Ai

Propose and Attend: Training-free MLLM Grounding Confidence via Multi-Token Localized Attention

arXiv:2607.05978v1 Announce Type: cross Abstract: Multimodal large language models can emit localized predictions, bounding boxes for objects and temporal windows for video and audio events, but they

local-aiarxiv-cs-ai

Agents

RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent

arXiv:2406.07089v4 Announce Type: replace Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have shown promise for remote sensing tasks such as visual question answering and scene

agentsarxiv-cs-cv

Segmentation before Answering: Pixel Grounding for MLLM Visual Reasoning

arXiv:2607.05798v1 Announce Type: cross Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have evolved from static perception to interleaved visual-language reasoning, often re

researcharxiv-cs-ai

Beyond Scene Priors: Fine-Grained Traffic Scene Reasoning with Benchmarking and Query-Guided Small-Object Focus

arXiv:2607.04149v1 Announce Type: new Abstract: In safety-critical traffic scenarios, answering complex questions relies on minute, localized visual cues. However, standard Multimodal Large Language M

Local Ai

DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences

arXiv:2607.02551v1 Announce Type: cross Abstract: Video multimodal large language models have made strong progress on open-ended video understanding, but they still lack precise local spatiotemporal p

local-aiarxiv-cs-ai

Tutorials

DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation

arXiv:2607.04779v1 Announce Type: new Abstract: Reasoning segmentation aims to predict pixel-wise masks for targets given complex language queries. Existing approaches leverage Multimodal Large Langua

tutorialsarxiv-cs-cv

DiCE-CIR: Direct Composition Learning for Efficient Zero-Shot Composed Image Retrieval

arXiv:2607.04665v1 Announce Type: new Abstract: Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image from a multimodal query consisting of a reference image and an edit text des

Agents

Do GUI Agents Believe Their Eyes? Diagnosing State-Belief Reliance on Pixels versus Structure

arXiv:2607.04334v1 Announce Type: new Abstract: Multimodal GUI agents read an interface through two redundant channels: the rendered pixels of a screenshot and a serialized structure such as a DOM or

agentsarxiv-cs-ai

DynaVieW: Schema-Guided World Modeling for Understanding Hierarchical Visual Dynamics

arXiv:2607.04112v1 Announce Type: cross Abstract: Multimodal LLMs struggle to systematically model the temporal evolution of visual scenes in videos or multi-image sequences. Such inputs require model

researcharxiv-cs-ai

Embodied Operators and Benchmarking: Toward Reusable and Deployable Embodied Intelligence Systems

arXiv:2607.03283v1 Announce Type: new Abstract: Embodied intelligence systems require not only end-to-end policy models, but also reusable functional modules that transform multimodal observations, ro

Evaluating and Understanding Model Editing for Medical Vision Language Models

arXiv:2607.05310v1 Announce Type: new Abstract: Model editing promises a fast, targeted way to correct post-deployment mistakes in medical vision-language models (VLMs) without costly retraining. Howe

FairFlow: Demystifying and Mitigating Stereotype Bias in Text-to-Image Diffusion Transformers

arXiv:2607.03180v1 Announce Type: new Abstract: Multimodal diffusion transformers (MM-DiTs) have emerged as the prevalent backbone for modern text-to-image generation systems. However, they exhibit cr

GuideMe: Multi-Domain Task Guidance and Intervention in Streaming Video

arXiv:2607.02991v1 Announce Type: new Abstract: While multimodal Large Language Models (MLLMs) excel at offline video understanding, an interesting question of how far they are from serving as a real-

Hierarchical Evidence-Driven Reasoning for Long Document Understanding

arXiv:2607.04625v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly c

researcharxiv-cs-ai

Agents

Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoning

arXiv:2607.02588v1 Announce Type: cross Abstract: Multimodal large language models excel on short clips but struggle on hour-long videos in an online setting, where frames are processed incrementally

agentsarxiv-cs-ai

Latent Visual Cache for Video Reasoning

arXiv:2607.02607v1 Announce Type: cross Abstract: Video reasoning requires Large Multimodal Models (LMMs) to remain grounded in dense evidence, yet existing systems largely adopt 'read-once, generate-

safetyarxiv-cs-cl

MentalThink: Shaping Thoughts in Mental SVG World

arXiv:2607.03530v1 Announce Type: new Abstract: We introduce MentalThink, a visual-symbolic reasoning paradigm that equips Multimodal LLMs (MLLMs) with an executable mechanism for 'mental' visualizati

safetyarxiv-cs-ai

Local Ai

OpenGlass: A Sensing-Computing Split Architecture for Local MLLM-Driven Real-Time Visual Assistance

arXiv:2607.03213v1 Announce Type: cross Abstract: We present OpenGlass, an open-source, privacy-oriented, local-first system for low-latency multimodal visual assistance, with a primary focus on blind

local-aiarxiv-cs-ai

Overloading Large Vision-Language Models for Jailbreaking

arXiv:2607.02961v1 Announce Type: new Abstract: Large Vision-Language Models (LVLMs) exhibit remarkable vision-language capabilities and are increasingly deployed in real-world applications such as pe

PPE-Bench: A Benchmark for Evaluating MLLM Unlearning under Private-Public Entanglement

arXiv:2607.02897v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have shown strong capabilities, but they may memorize private information from web data, raising privacy conc

ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space

arXiv:2607.02907v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress but still struggle with complex visual reasoning tasks requiring multi-step

researcharxiv-cs-cl

SEAM: Smooth Execution of Action-Chunked Motion for Vision-Language-Action Policies

arXiv:2607.04609v1 Announce Type: new Abstract: Vision-Language-Action (VLA) policies that execute fixed-length action chunks can exhibit multimodal bifurcation: a cross-chunk inconsistency in which a

safetyarxiv-cs-ro

SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

arXiv:2511.07403v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language tasks, but continue to struggle with spatial rea

UI-MOPD: Multi-Platform On-Policy Distillation for Continual GUI Agent Learning

arXiv:2607.04425v1 Announce Type: cross Abstract: Recent advances in multimodal foundation models and agent systems have driven GUI agents from single-platform task execution toward cross-platform int

safetyarxiv-cs-ai

VCB Bench: An Evaluation Benchmark for Audio-Grounded Large Language Model Conversational Agents

arXiv:2510.11098v5 Announce Type: replace-cross Abstract: Recent advances in large audio language models (LALMs) have greatly enhanced multimodal conversational systems. However, existing benchmarks r

What if a routine breast cancer H&E slide could help answer a hard question: how likely is this cancer to come back? Our @NatureComms paper …

What if a routine breast cancer H&E slide could help answer a hard question: how likely is this cancer to come back? Our @NatureComms paper introduces a multimodal AI test for recurrence risk across i

researchyann-lecun--x

6 Jul 2026

A global predicted-fMRI drive signal from TRIBE does not predict YouTube replay heatmaps

model-releasesarxiv-cs-lg

arXiv:2607.01400v1 Announce Type: cross Abstract: Deep multimodal brain-encoding models now predict fMRI responses to naturalistic video with high accuracy. Whether their predicted neural signals also

NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative Understanding

model-releasesarxiv-cs-lg

arXiv:2601.01095v3 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have achieved impressive progress in vision-language reasoning, yet their ability to understand tempo

SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models

arXiv:2607.01876v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have achieved remarkable progress in multimodal understanding, yet their enormous parameter scale and cross-modal

Tutorials

Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learning

arXiv:2607.02490v1 Announce Type: new Abstract: Large vision-language models can reason over multimodal inputs by generating textual chains of thought (CoT). A key capability exhibited in CoT reasonin

tutorialsarxiv-cs-cl

AV-SyncBench: Decoupled Benchmarking of Temporal and Semantic Audio-Visual Synchronization

arXiv:2607.00726v1 Announce Type: new Abstract: Audio-visual feature extraction is a fundamental component of multimodal understanding and generation tasks. However, existing evaluation protocols for

GeoSearcher: Anchor-Guided Progressive Reasoning for Remote Sensing Visual Grounding with Process Supervision

arXiv:2607.01050v1 Announce Type: new Abstract: Recent multimodal large language models (MLLMs) have shown strong cross-modal understanding and coordinate generation abilities in visual grounding. How

Information-Regularized Attention for Visual-Centric Reasoning

arXiv:2607.00434v1 Announce Type: new Abstract: Vision-language models (VLMs) have become a paradigm for multimodal learning, yet remain unstable due to object hallucination, weak visual grounding, an

Applications

Learning When to Listen: Gated Affect Fusion for Human Motion Prediction

arXiv:2607.00296v1 Announce Type: cross Abstract: Human motion forecasting in unconstrained real-world videos remains challenging due to the ambiguity of future behaviors and the presence of noisy mul

applicationsarxiv-cs-ai

Linguistic Relative Policy Optimization for Video Anomaly Reasoning

arXiv:2607.00654v1 Announce Type: new Abstract: Video anomaly detection (VAD) with multimodal large language models has shown strong potential, yet most existing methods still depend on large-scale an

MindAU: EEG-Conditioned Facial Action Unit Editing via Dual-Stream Manifold Alignment

arXiv:2607.00410v1 Announce Type: new Abstract: Recent brain decoding studies have made substantial progress in reconstructing externally perceived visual content from neural signals. However, using e

RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarios

arXiv:2511.18011v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have demonstrated powerful capabilities in general spatial understanding and reasoning. However, their fine

StochasT: Learning with Stochastic Turn Depth for Visual Instruction Tuning

arXiv:2607.00465v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) rely extensively on Visual Instruction Tuning (VIT) to elicit their multimodal reasoning capabilities. However, w

Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptions

arXiv:2507.15692v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) provide new opportunities for blind and low vision (BLV) people to access visual information in their daily l

safetyarxiv-cs-cl

Synergistic Perception-Reasoning Governance: Grounding Medical MLLMs with Verifiable Anatomical Evidence

arXiv:2607.00060v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) show strong promise for clinical VQA and radiology report generation, yet inference-time hallucinations still u

researcharxiv-cs-cv

Towards High-Resolution Visual Perception via Hierarchical Entity Exploration

arXiv:2607.00816v1 Announce Type: new Abstract: High-resolution (HR) image perception remains a key challenge in multimodal large language models (MLLMs), as fine-grained details are often lost when t

AMALIA-VL: A Native European Portuguese Open-Source Vision and Language Model

arXiv:2606.19100v2 Announce Type: replace Abstract: Large Vision and Language Models (LVLMs) have advanced rapidly, yet European Portuguese (pt-PT) remains systematically underserved by existing open-