Search: “multimodal”

ArtECulture: Benchmarking Culture-Conditioned Visual Emotion Understanding in Multimodal Large Language Models

model-releasesarxiv-cs-cl

arXiv:2608.03358v1 Announce Type: new Abstract: Existing visual emotion understanding methods typically ignore cultural variations in emotional perception. We introduce culture-conditioned visual emot

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Evidence-Grounded Multimodal Knowledge Graph Construction for Multi-Lecture Educational Reasoning

DGX agent

arXiv:2608.03161v1 Announce Type: new Abstract: Lecture videos distribute knowledge across speech, slide text, diagrams, equations, and presentation order, which transcript-only retrieval does not ful

researcharxiv-cs-ai

5 Aug 2026

LocAnyMed: Vision-Language Grounding for Multimodal Medical Images

arXiv:2608.03322v1 Announce Type: new Abstract: Medical visual grounding connects free-form clinical queries to spatial evidence in medical images and is an important component of interpretable medica

Multimodal Auto-regressive Transformer Surrogate for Modeling Variable Operations and Quantifying Uncertainty in Geological Carbon Storage

arXiv:2608.02629v1 Announce Type: cross Abstract: The use of variable well perforation and injection strategies can improve the efficiency of geological carbon storage operations. We develop a new mul

Multimodal Plant Root Phenotyping with Integration of 3D Skeleton Extraction and Language Analysis

arXiv:2608.03109v1 Announce Type: new Abstract: Plant root phenotyping is fundamental to understanding below-ground structures, optimizing crop management, and improving agricultural sustainability. T

Applications

Implicit Neural Representations for Multimodal Longitudinal Image Imputation and Interpolation

arXiv:2608.02324v1 Announce Type: new Abstract: Longitudinal multiparametric MRI is central to follow-up imaging in oncology, yet real-world clinical data are characterised by missing sequences, heter

applicationsarxiv-cs-cv

4 Aug 2026

New York Smells: A Large Multimodal Dataset for Olfaction

arXiv:2511.20544v2 Announce Type: replace Abstract: While olfaction is central to how animals perceive the world, this rich chemical sensory modality remains largely inaccessible to machines. One key

4 Aug 2026

SPARC-Rad: A Multimodal Benchmark Dataset and Evaluation Pipeline for Spatial and Anatomical Reasoning in Radiology Vision-Language Models

arXiv:2608.00100v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly being evaluated for medical imaging, but many available benchmarks emphasize disease classification, repo

4 Aug 2026

Digital Harf: A Clinically Integrated Multimodal AI System for Pervasive Arabic Speech and Language Therapy

arXiv:2607.27212v1 Announce Type: cross Abstract: Children with Autism Spectrum Disorder in Arabic-speaking countries face compounded barriers to effective speech and language therapy: a shortage of q

safetyarxiv-cs-cl

31 Jul 2026

A Closer Look at Dynamic Scene Graph Generation In the Era of Multimodal Large Language Models

arXiv:2503.15846v2 Announce Type: replace Abstract: Dynamic Scene Graph Generation (DSGG) aims to capture objects and their evolving relations in videos. Despite recent progress, the practicality and

30 Jul 2026

OmniAD: Detect and Understand Industrial Anomaly via Multimodal Reasoning

arXiv:2505.22039v2 Announce Type: replace Abstract: While anomaly detection has made significant progress, generating detailed analyses that incorporate industrial knowledge remains a challenge. To ad

30 Jul 2026

What Can Latent World Models Know? Physical Parameter Identifiability in Multimodal Predictive Representations

model-releasesarxiv-cs-lg

arXiv:2607.27017v1 Announce Type: new Abstract: A central premise of latent world models is that predicting the future forces a representation to internalize the physics of its environment. Which phys

30 Jul 2026

DisasterTD: Disaster Toponym Disambiguation Using Multimodal LLMs and Cross-View Geolocalization

arXiv:2607.24856v1 Announce Type: cross Abstract: Social media imagery (SMI) provides timely and fine-grained ground perspectives that are valuable for situational awareness and emergency response. Un

FinAbstain: Uncertainty-Calibrated Multimodal RAG for Selective Financial Forecasting

arXiv:2607.24875v1 Announce Type: new Abstract: Large language models (LLMs) can synthesize financial narratives but may express high confidence when evidence is sparse, stale, or contradictory. This

researcharxiv-cs-lg

Food Image Segmentation with LLM-Derived Ingredient Labels and Multimodal Fusion

arXiv:2607.25820v1 Announce Type: new Abstract: Food image segmentation plays a vital role in health-related applications such as nutrition tracking and personalized health monitoring. However, existi

LaP-Forensics: Latent-Pixel Consistency Guided Multimodal Reasoning for Deepfake Detection

arXiv:2607.25962v1 Announce Type: new Abstract: Recent generative models can produce images with few obvious visual artifacts, weakening detectors and explanations that rely only on surface appearance

Towards Reliable Stain Transfer: An Iterative Data-Model Co-Optimization Framework Based on Multimodal Expert-Guided Assessment

arXiv:2607.25393v1 Announce Type: new Abstract: Histopathological examination primarily relies on hematoxylin and eosin (H&E) and immunohistochemistry (IHC) staining. Although IHC provides critical mo

DriveDNA: A Large-Scale Multimodal Naturalistic Driving Dataset and Benchmark for Driving Style Identification

model-releasesarxiv-cs-lg

arXiv:2607.23822v1 Announce Type: new Abstract: Driving style captures stable, driver-specific patterns in how a vehicle is driven. In naturalistic data, however, this signal is hard to isolate becaus

28 Jul 2026

GaitFace: A Multimodal Dataset for Long-Range Person Identification

arXiv:2607.23542v1 Announce Type: new Abstract: Efficient border control is becoming a significant global challenge, mainly due to severe congestion and extended passenger waiting times. To mitigate t

28 Jul 2026

Local Ai

Harnessing X-ray Absorption Spectroscopy Data through Multimodal Mining of Battery Literature

arXiv:2607.23886v1 Announce Type: cross Abstract: X-ray absorption spectroscopy (XAS) is central to understanding the local electronic and atomic structure of materials, yet most published spectra rem

local-aiarxiv-cs-ai

28 Jul 2026

Autoregressive EHR Foundation Models with Multimodal Inputs

arXiv:2607.22264v1 Announce Type: new Abstract: Autoregressive foundation models trained on tokenized electronic health records (EHRs) can support zero-shot clinical prediction, yet most operate on st

safetyarxiv-cs-lg

27 Jul 2026

Benchmarking Fine-tuning and Retrieval Strategies for a Multimodal Language Model on the NRC Reactor Operator Licensing Examination

model-releasesarxiv-cs-cl

arXiv:2607.22067v1 Announce Type: new Abstract: The integration of large language models (LLMs) into the nuclear power industry requires outputs grounded in domain-specific knowledge. This study evalu

27 Jul 2026

Local Multimodal Music Alignment from Global Supervision

arXiv:2607.10023v2 Announce Type: replace-cross Abstract: Understanding music requires understanding localized relationships across data modalities, e.g., how time in performance audio maps onto posit

safetyarxiv-cs-lg

27 Jul 2026

Human-Inspired Framework for Robotic Craniotomy: Integrating Multimodal Fusion and Adaptive Trajectory Adjustment

arXiv:2607.21058v1 Announce Type: new Abstract: Manual craniotomy is a high-risk, skill-dependent procedure associated with surgeon fatigue and potential dural injury. While robotic approaches have im

safetyarxiv-cs-ro

Multimodal Learning for Arcing Detection in Pantograph-Catenary Systems

arXiv:2602.08792v2 Announce Type: replace-cross Abstract: The pantograph-catenary interface is essential for ensuring uninterrupted and reliable power delivery in electrified rail systems. However, el

RL-MACRO: A Cybernetic Closed-Loop Intelligence Framework for Multimodal Adaptive Robotic Craniotomy

arXiv:2607.21113v1 Announce Type: new Abstract: Autonomous robotic craniotomy requires continuous regulation of tool-tissue interactions to mitigate mechanical overload and thermal damage while mainta

safetyarxiv-cs-ro

Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning

arXiv:2607.21300v1 Announce Type: cross Abstract: Machine unlearning has emerged as a tool for removing personal data from trained models to comply with recent AI regulations. To evaluate unlearning e

HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement

arXiv:2607.18217v2 Announce Type: replace Abstract: Human-object centric video personalization (HOCVP) is a core task within subject-driven video generation. However, existing methods suffer from two

safetyarxiv-cs-cv

In-Context Learning for Wound Classification with Small Multimodal Language Models

arXiv:2607.18819v1 Announce Type: new Abstract: Wound image classification is often treated as a task-specific supervised learning problem, requiring substantial amounts of manually labelled data and

Leveraging ChatGPT's Multimodal Vision Capabilities to Rank Satellite Images by Poverty Level: Advancing Tools for Social Science Research

arXiv:2501.14546v2 Announce Type: replace-cross Abstract: This paper investigates the novel application of Large Language Models (LLMs) with vision capabilities to analyze satellite imagery for villag

Memory-Augmented Multimodal Large Language Models for Small Object Understanding in Streaming Aerial Videos

arXiv:2607.19857v1 Announce Type: cross Abstract: Language-guided aerial perception aims to understand user-specified tiny targets in complex unmanned aerial vehicle (UAV) scenes. In real UAV deployme

A novel unsupervised machine learning strategy to handle multimodal cardiac PET/MRI data

arXiv:2607.13936v1 Announce Type: new Abstract: Arrhythmogenic left ventricular cardiomyopathy is a genetic myocardial disease difficult to diagnose due to the lack of gold standard criteria. Simultan

Kepler-Encoder-v0.1: Towards a Multimodal Embedding Model for Robots

arXiv:2607.13522v1 Announce Type: cross Abstract: A robot must understand the state of its own body, but a camera sees only part of it. Force and contact leave almost no trace in a single frame, and r

Agents

M^ext{4}World: A Multi-view Multimodal Driving World Model for Interactive Object Manipulation and Minute-long Streaming

arXiv:2607.14005v1 Announce Type: new Abstract: Driving-world generation has emerged as a core capability for scalable autonomous-driving simulation, yet existing methods remain limited in object-leve

agentsarxiv-cs-cv

Local Ai

Multimodal Assessment of Pancreatic Cancer Resectability Using Deep Learning

arXiv:2607.13826v1 Announce Type: cross Abstract: Accurate determination of pancreatic ductal adenocarcinoma (PDAC) resectability relies on evaluating how the tumor interacts with major peripancreatic

local-aiarxiv-cs-ai

Auditing Data Leakage in Whole-Slide Image Multimodal Benchmarks

arXiv:2607.12278v1 Announce Type: new Abstract: Recent vision-language models (VLMs) for computational pathology report striking zero-shot performance on whole-slide image (WSI) visual question answer

15 Jul 2026

GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model

arXiv:2607.06882v1 Announce Type: cross Abstract: Visual navigation policies built on large pretrained models have so far followed a common recipe: a dedicated visual encoder, a bespoke action head, a

safetyarxiv-cs-ai

9 Jul 2026

Applications

Multimodal Spatiotemporal-Frequency Fusion with Peak Enhancement for Cellular Traffic Forecasting

arXiv:2607.07016v1 Announce Type: cross Abstract: Accurate forecasting of cellular network traffic is essential for network planning, resource allocation, and quality-of-service assurance in modern mo

applicationsarxiv-cs-ai

9 Jul 2026

Overview of the NLPCC 2026 Shared Task 1: Difficulty-Aware Multilingual and Multimodal Medical Instructional Video Understanding Evaluation

arXiv:2607.06618v1 Announce Type: cross Abstract: Following the CMIVQA, MMI-VQA, and M4IVQA challenges in NLPCC 2023--2025, we introduce the Difficulty-Aware Medical Instructional Video Question Answe

9 Jul 2026

GaussFusion: Towards Multimodal 3D Gaussian Pretraining

arXiv:2607.05906v1 Announce Type: new Abstract: 3D Gaussian Splatting provides an explicit representation that jointly models geometry and appearance, serving as a scalable foundation for 3D represent

safetyarxiv-cs-cv

8 Jul 2026

Local Ai

A Multimodal Reasoning Typology for Grounding Chart-Image Coherence in Science Communication

arXiv:2607.05222v1 Announce Type: new Abstract: Charts and images appear together throughout scientific publications, yet most computational work does not characterize their coherence. We argue that a

local-aiarxiv-cs-cv

Agents

AppAgent: Multimodal Agents as Smartphone Users

arXiv:2312.13771v3 Announce Type: replace Abstract: Recent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks. This paper i

agentsarxiv-cs-cv

BanglaMemeEvidence: A Multimodal Benchmark Dataset for Explanatory Evidence Detection in Bengali Memes

arXiv:2607.03981v1 Announce Type: cross Abstract: Memes have become influential communication tools on social media, combining viral visuals with concise messaging to convey impactful ideas. While sub

Applications

From Geometric Labels to Semantic Understanding of Indoor Building Components Using Multimodal Large Language Models

arXiv:2607.03661v1 Announce Type: new Abstract: Point cloud-based understanding has become an important enabler for facility operation and maintenance involving indoor building components. However, ex

applicationsarxiv-cs-cv

Glance-Say: Multimodal Human-Robot Collaboration and Intent Recognition via Sticky Glance

arXiv:2603.06121v2 Announce Type: replace Abstract: Gaze and speech are promising interaction modalities for individuals with motor impairments, yet robust intent recognition in multi-object environme

researcharxiv-cs-ro

Jointly Improving Dialect Identification and ASR in Indian Languages using Multimodal Feature Fusion

arXiv:2607.02862v1 Announce Type: new Abstract: Automatic Speech Recognition (ASR) and Dialect Identification (DID) are crucial for Indian languages, many of which are low-resource and exhibit signifi

researcharxiv-cs-cl

K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos

arXiv:2607.02680v1 Announce Type: cross Abstract: MLLMs have shown strong zero-shot capabilities across diverse inputs such as across images, video, audio, and text. A crucial, yet underexplored, appl

Learning Taxonomic Trees with Hierarchical Representation Regularization for Large Multimodal Models

arXiv:2607.02909v1 Announce Type: cross Abstract: Taxonomies provide key information about the semantic relationships between concepts and the inherent organization of vision and language. Despite the