Search: “multimodal”

Visual Credit Audit for Multimodal Spatial Reasoning

arXiv:2607.27069v1 Announce Type: new Abstract: Closed yes/no spatial benchmarks can reward a correct answer even when the image adds little support beyond no-image contexts. Under a fixed forced-choi

30 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning

DGX agent

arXiv:2607.24424v1 Announce Type: new Abstract: Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features ca

model-releasesarxiv-cs-cv

28 Jul 2026

Tutorials

Multimodal Semantic-Probabilistic Objectness for Open World Object Detection

arXiv:2607.23981v1 Announce Type: cross Abstract: Open-world object detection (OWOD) requires a detector to recognize known categories, discover unnamed objects from unseen categories, and incremental

tutorialsarxiv-cs-ai

28 Jul 2026

ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness

arXiv:2607.23537v1 Announce Type: new Abstract: Autonomous driving under adverse weather remains a critical challenge, yet existing vision-language benchmarks mainly evaluate under standard conditions

28 Jul 2026

Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms

arXiv:2607.21780v1 Announce Type: new Abstract: Document packets, multiple documents concatenated into a single file, are common in government and administrative workflows, yet splitting them into the

27 Jul 2026

ETPDesigner: Multi-Agent Orchestration for Interactive Multimodal Electronic Theater Program

arXiv:2607.19947v1 Announce Type: new Abstract: Electronic Theater Programs (ETPs) serve as critical promotional media in the performing arts, comprising a multi-page collection of heterogeneous visua

23 Jul 2026

FM^2: Unified Federated Foundation Models for Heterogeneous Multimodal Medical Imaging

arXiv:2607.13386v1 Announce Type: new Abstract: Building foundation models for medical imaging requires pooling data across institutions, yet privacy regulations prohibit centralized aggregation. Exis

16 Jul 2026

Multimodal Empirical Bayes Variational Autoencoders for Joint Longitudinal and Time-to-Event Modeling

arXiv:2607.13984v1 Announce Type: cross Abstract: Longitudinal tumor measurements, dropout information, and genetic covariates provide complementary information about treatment response, but integrati

researcharxiv-cs-lg

16 Jul 2026

Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Models

arXiv:2607.08317v1 Announce Type: new Abstract: Modern AI models achieve strong performance on many established benchmarks, yet they still fail on tasks that humans find almost trivial, such as manipu

10 Jul 2026

MatBind: A Shared Embedding Space for Multimodal Materials Characterization

arXiv:2607.08470v1 Announce Type: new Abstract: Fully characterizing a crystalline material requires integrating heterogeneous data sources -- atomic structures, diffraction patterns, electronic densi

safetyarxiv-cs-lg

10 Jul 2026

BUS: Brain-Inspired Unsupervised Self-Reflection for Advanced Multimodal Reasoning

arXiv:2607.07361v1 Announce Type: new Abstract: Current Vision-Language Models (VLMs) often struggle to handle complex visual tasks that require consistent and fine-grained reasoning. Recent methods a

9 Jul 2026

CAIRN: Cross-Room 3D Scene Understanding with Topology-Aware Large Multimodal Models

arXiv:2607.06534v1 Announce Type: new Abstract: Existing 3D scene-grounded Large Language Models (3D-LLMs) focus on answering questions grounded in simplified single-room 3D scenes, lacking the abilit

8 Jul 2026

LEGATO 2: Toward Multimodal Sheet Music Recognition and Understanding

arXiv:2607.05769v1 Announce Type: cross Abstract: We propose a novel pipeline, Legato 2, for extracting symbolic notation and semantic knowledge from images of sheet music. Legato 2 features the first

8 Jul 2026

Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards

arXiv:2607.04727v1 Announce Type: cross Abstract: Automatic data visualization generation has advanced rapidly with multi-modal large language models, yet existing efforts largely focus on static char

Explainable AI for Screening Abuse-Related Trauma in Bangladeshi Children: A Training-Free Multimodal Framework Evaluated on Noise-Aware Synthetic Data

arXiv:2607.04010v1 Announce Type: new Abstract: Bangladesh has an estimated 1.17 mental-health professionals per 100,000 population and only six child psychiatrists nationwide. No Bengali-language, cu

Integrating Neural Encoders in Bayesian Generalized Linear Mixed Models for Multimodal Data

model-releasesarxiv-cs-lg

arXiv:2607.04647v1 Announce Type: cross Abstract: Scalable Bayesian inference for generalized linear mixed models (GLMMs) provides uncertainty-aware analysis of correlated longitudinal data, but exist

Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Remote Sensing

arXiv:2607.03644v1 Announce Type: cross Abstract: Decades of orbital missions have produced multi-modal remote sensing data for the Moon, spanning optical imagery, spectroscopy, thermal emission, rada

OmniLayout: A Schematic-Coupled Multimodal Benchmark for Constraint-Aware Geometric Reasoning in PCB Layout

arXiv:2607.03261v1 Announce Type: new Abstract: Recent large language models (LLMs) have demonstrated remarkable progress in 3D spatial reasoning, spatial grounding, and fine-grained geometric underst

Agents

SurgAM: Surgical Affordance Map Prediction with Multimodal Feature Fusion for Robot Autonomy

arXiv:2607.04378v1 Announce Type: new Abstract: Surgical automation is being increasingly studied, yet bridging visual scene understanding with autonomous action planning remains a fundamental challen

agentsarxiv-cs-ro

ViPo-MLLM: Visual-Pose Multimodal LLM for Gloss-Free Sign Language Translation

arXiv:2607.03657v1 Announce Type: cross Abstract: Gloss-free Sign Language Translation (SLT) translates sign language videos into spoken-language sentences without gloss annotations, avoiding costly l

PairCoder++: Pair Programming as a Universal Paradigm for Verified Code-Driven Multimodal and Structured-Artifact Generation

arXiv:2607.01883v1 Announce Type: new Abstract: Code is the medium through which large language models generate structured artifacts: charts, scientific figures, vector graphics, CAD models, 3D scenes

3 Jul 2026

From Multimodal Perception to Strategic Reasoning: A Survey on AI-Generated Game Commentary

arXiv:2506.17294v3 Announce Type: replace-cross Abstract: The advent of artificial intelligence has propelled AI-Generated Game Commentary (AI-GGC) into a rapidly expanding research area, offering adv

1 Jul 2026

Attribution Graphs and Causal Probing for Mechanistic Discovery and Bias Repair in Multimodal Generative Learning

arXiv:2510.12957v4 Announce Type: replace-cross Abstract: We treat the internals of generative models as mechanistic objects rather than black boxes. We introduce extbf{Attribution Graphs} (AGs), whic

Data-Efficient Multimodal Alignment for Histopathology-based Molecular Prediction

arXiv:2606.29949v1 Announce Type: cross Abstract: H&E-stained whole-slide images offer cohort-scale availability and rich spatial context but lack molecular specificity, whereas bulk RNA-seq provides

safetyarxiv-cs-ai

EVLA: An Electro-Aware Multimodal Assistant for Physically-Grounded Driving Reasoning and Control

arXiv:2606.28938v1 Announce Type: new Abstract: Modern vision-language models (VLMs) for driving assistants typically treat vehicle dynamics as a black box, resulting in decisions that lack awareness

LLM-Guided Planning for Multi-hop Reasoning over Multimodal Nuclear Regulatory Documents

arXiv:2606.29399v1 Announce Type: new Abstract: Reviewing nuclear regulatory documents requires multi-hop reasoning across tens of thousands of pages, where judgments depend on evidence assembled acro

Unlocking the Visual Record of Materials Science: A Large-Scale Multimodal Dataset from Scientific Literature

arXiv:2606.29667v1 Announce Type: cross Abstract: The materials science literature encodes decades of experimental knowledge in figures, yet this visual record remains locked away and inaccessible to

PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception

arXiv:2606.28322v1 Announce Type: new Abstract: We introduce PerceptionRubrics, a rubric-based evaluation framework that addresses the gap between saturated benchmark scores and real-world brittleness

29 Jun 2026

RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning

arXiv:2606.28266v1 Announce Type: new Abstract: Remote Sensing Image Change Captioning (RSICC) aims to describe changes between bi-temporal remote sensing images and holds significant research and app

29 Jun 2026

HarmVideoBench: Benchmarking Harmful Video Understanding in Large Multimodal Models

arXiv:2606.27187v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) have recently shown immense potential in automated content moderation, sparking growing interest in developing ha

26 Jun 2026

Hardware

OctoSense: Self-Supervised Learning for Multimodal Robot Perception

arXiv:2606.27317v1 Announce Type: new Abstract: We present OctoSense, an open-source sensor platform with stereo RGB and event cameras, LiDAR, a thermal camera, an inertial measurement unit, RTK-corre

hardwarearxiv-cs-cv

26 Jun 2026

Context-Aware Prediction of Student Quiz Performance with Multimodal Textbook Features

arXiv:2606.24770v1 Announce Type: cross Abstract: Educational platforms often predict student performance from prior interactions, but the assessment content itself also varies in linguistic and visua

24 Jun 2026

Navigating User Behavior toward Personalized Multimodal Generation

arXiv:2606.24196v1 Announce Type: new Abstract: Modern AIGC pipelines deliver high-fidelity images and videos but presuppose a well-formed creation instruction, while end users rarely articulate visua

24 Jun 2026

PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Models

arXiv:2606.24388v1 Announce Type: new Abstract: We introduce a large-scale, open-source dataset of pre-generated adversarial attacks for vision-language models (VLMs). The dataset is designed to be di

safetyarxiv-cs-ai

24 Jun 2026

A Multimodal Tiltwing Framework for Bioinspired Aerial Robots

arXiv:2606.22046v1 Announce Type: new Abstract: Tailless flapping-wing micro-aerial vehicles (FWMAVs) mimic the impressive flight performance of hummingbirds, utilising unsteady aerodynamic effects. H

researcharxiv-cs-ro

GIM-ENDO: A Multimodal Endoscopic Image and Video Dataset for Gastric Intestinal Metaplasia Morphology and Pathology

arXiv:2606.20919v1 Announce Type: new Abstract: Gastric intestinal metaplasia (GIM) is a precursor lesion to gastric dysplasia and adenocarcinoma whose early detection is crucial for intervening in th

MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planning

arXiv:2606.20641v1 Announce Type: cross Abstract: Multi-modal Large Language Models (MLLMs) have demonstrated remarkable capabilities in semantic understanding and common sense reasoning, making them

safetyarxiv-cs-lg

Multimodal Image Colorization: Quantifying the Impact of Text-Conditioned Guidance on Grayscale-to-Color Translation

arXiv:2606.20722v1 Announce Type: cross Abstract: Grayscale images are commonly found in historical photography restoration, medical imaging, and artistic media. However, automatically applying color

VeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instruct

arXiv:2606.23543v1 Announce Type: cross Abstract: Scaling reinforcement learning for visual mathematical reasoning requires more than generating harder questions: as data volume grows, the reward labe

Applications

DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action model

arXiv:2606.12105v1 Announce Type: cross Abstract: Vision-language-action (VLA) models inherit a shared synchronous clock from vision-language pretraining, processing every input at one rate. This is m

applicationsarxiv-cs-cv

11 Jun 2026

GLACIER: A Multimodal Student-Teacher Foundation Model for Molecular Property Prediction

arXiv:2606.11382v1 Announce Type: new Abstract: Deep learning models facilitate the discovery of molecules with tailored properties among billions of candidate compounds. However, the computational bu

safetyarxiv-cs-lg

11 Jun 2026

A Multimodal RGB and Events Dataset for Hand Detection in First-Person View

arXiv:2606.10790v1 Announce Type: new Abstract: Existing hand detection algorithms work on images and the detection rate is restricted by the frame rate of the camera. In hand detection applications f

10 Jun 2026

GD-MIL: Grade-Disentangled Multiple Instance Learning for Multimodal Biochemical Recurrence Prediction in Prostate Cancer

arXiv:2606.09453v1 Announce Type: new Abstract: Biochemical recurrence (BCR) after radical prostatectomy is a critical endpoint in prostate cancer, yet risk stratification relies almost entirely on va

9 Jun 2026

PereStruct: Multimodal Semantic Assembly for Robust Historical Document Parsing

arXiv:2606.07661v1 Announce Type: new Abstract: Parsing historical documents with complex, non-standard layouts remains a fundamental bottleneck in large-scale archival digitization. Unlike modern typ

9 Jun 2026

Toward Scalable Co-located Practical Learning: Assisting with Computer Vision and Multimodal Analytics

arXiv:2603.13679v2 Announce Type: replace-cross Abstract: Co-located practical learning leaves evidence in visible actions around patients, task resources and room zones, but these traces are often re

9 Jun 2026

Tutorials

EgoPressDiff: Multimodal Video Diffusion for Egocentric UV-Domain Hand-Pressure Estimation

arXiv:2606.06872v1 Announce Type: cross Abstract: Estimating hand-surface contact pressure from an egocentric view is crucial for AR/VR devices, robotic imitation, and ergonomic analysis. Existing met

tutorialsarxiv-cs-ai

8 Jun 2026

GuideCAD: A Lightweight Multimodal Framework for 3D CAD Model Generation via Prefix Embedding

arXiv:2606.07024v1 Announce Type: new Abstract: Multi-modal approaches used for 3D CAD generation require substantial computational resources, necessitating efficient training. To address this, we pro

8 Jun 2026

Step-adaptive multimodal fusion network with multi-scale cloud feature learning for ultra-short-term solar irradiance forecasting

arXiv:2606.06102v1 Announce Type: new Abstract: Ultra-short-term solar irradiance prediction is critical for photovoltaic system dispatch and power grid stability. Existing approaches suffer from thre