Search: “multimodal”

Pose-Aware Multimodal Automatic Tagging for Greek Traditional Music

arXiv:2609.27094v1 Announce Type: cross Abstract: Automatic tagging is a core task in Music Information Retrieval (MIR), yet most tagging systems exploit only audio. Live music performance is inherent

24 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

ChemVTS-Bench: Evaluating Visual-Textual-Symbolic Reasoning of Multimodal Large Language Models in Chemistry

DGX agent

arXiv:2511.17909v2 Announce Type: replace Abstract: Chemical reasoning inherently integrates visual, textual, and symbolic modalities, yet existing benchmarks rarely capture this complexity, often rel

model-releasesarxiv-cs-ai

23 Sept 2026

FMMD: A multimodal multidisciplinary dataset of open peer reviews from F1000Research

arXiv:2602.14285v2 Announce Type: replace-cross Abstract: Automated scholarly paper review (ASPR) has entered the coexistence phase with traditional peer review, where artificial intelligence (AI) sys

23 Sept 2026

This is both fun and a useful measure of how much better multimodal AI has gotten In a short time.

model-releasesethan-mollick--x

This is both fun and a useful measure of how much better multimodal AI has gotten In a short time. Can AI tell if you've built your IKEA furniture wrong? Our new benchmark, the Furniture Assembly Benc

23 Sept 2026

Agents

Automatic multimodal UX improvement recommendations from LLM agent user simulations

arXiv:2609.22971v1 Announce Type: new Abstract: Evaluating user experience (UX) on live websites through user testing is expensive, subjective, and difficult to scale. LLM agents offer a promising rou

agentsarxiv-cs-cl

Beyond the Stitching Assumption: A Unified Framework for Multimodal Synthetic Data Evaluation via Semantic Quantization

arXiv:2609.22149v1 Announce Type: new Abstract: Multimodal synthetic datasets combine structured attributes with free text, but are often evaluated separately. Such metrics can remain high after tabul

researcharxiv-cs-cl

GLR-MM: Graph-Based Global-Local Reconstruction for Robust Multimodal Chest X-ray and EHR Representation Learning under Missing Modalities

arXiv:2609.23876v1 Announce Type: new Abstract: Clinical multimodal models must often predict before all chest X-ray (CXR) and electronic health record (EHR) inputs are available. Existing approaches

researcharxiv-cs-lg

Preoperative Prediction of Microvascular Invasion in Hepatocellular Carcinoma by Integrating Multimodal Ultrasound and Clinical Data: A Multicenter Study

arXiv:2609.24524v1 Announce Type: new Abstract: Background: Microvascular invasion (MVI) predicts recurrence and survival in hepatocellular carcinoma (HCC) but requires postoperative histopathology fo

Representation-guided in-context learning for medical image interpretation with multimodal large language models

model-releasesarxiv-cs-cl

arXiv:2609.24057v1 Announce Type: cross Abstract: Medical image interpretation is central to diagnosis and care, yet adapting general-purpose multimodal large language models (MLLMs) often requires re

SAGG: Sample-Adaptive Gradient Gating for Robust Multimodal Learning under Heterogeneous Corruption

model-releasesarxiv-cs-lg

arXiv:2609.20302v1 Announce Type: new Abstract: Multimodal gradient balancing methods modulate encoder gradients with a shared scalar per modality, implicitly assuming that corruption is uniform acros

18 Sept 2026

Uni-LaDiR: Latent Diffusion Unifies Multimodal Reasoning

arXiv:2609.19878v1 Announce Type: cross Abstract: Multimodal reasoning requires models to draw on information from multiple modalities throughout the reasoning process. Yet existing methods often conc

researcharxiv-cs-cl

18 Sept 2026

Applications

MCLC-NET: Multimodal Continual Learning for Leaf Counting

arXiv:2609.18129v1 Announce Type: cross Abstract: Leaf counting is an important task in plant phenotyping for monitoring plant growth and estimating crop yield. Most existing methods rely on RGB image

applicationsarxiv-cs-lg

17 Sept 2026

Efficient Multimodal Generative Recommendation with Latent Narrative Reasoning

model-releasesarxiv-cs-ai

arXiv:2609.16070v1 Announce Type: cross Abstract: Generative recommendation reformulates item prediction as semantic identifier generation, yet episodic content introduces a fundamentally different se

16 Sept 2026

What Do Hallucinations Reveal About Multimodal Reasoning? Diagnosing Visual Grounding Failures via Contrastive Decoding Probes

arXiv:2609.16646v1 Announce Type: new Abstract: When strong multimodal models are widely available, progress requires new scientific methodologies beyond benchmark scores---using models as instruments

16 Sept 2026

Query-Conditioned Spherical Centroid Aggregation for Multimodal Retrieval

arXiv:2609.15335v1 Announce Type: new Abstract: Multimodal retrieval integrates video, audio, subtitles, and text; however, recent geometric aggregators, such as Gramian volumes, hyperbolic volumes, a

15 Sept 2026

ReH-FUSE: Reliability-Aware Hierarchical Fusion of Experts for Multimodal Emotion Recognition in Conversation

arXiv:2609.13857v1 Announce Type: new Abstract: Multimodal emotion recognition in conversation (ERC) requires adapting to the instance-dependent reliability of different evidence sources. Lexical cont

researcharxiv-cs-lg

15 Sept 2026

Where Should Language Sit in a Multimodal Model? Lessons from What Language Does to Human Perception and Cognition

arXiv:2609.07474v2 Announce Type: replace Abstract: Language models compute over tokens: language is their input, their output, and increasingly their internal representation. Whether language should

15 Sept 2026

A Multimodal Explainable Deep Learning Framework for Alzheimer's Disease Diagnosis using 3D Magnetic Resonance Imaging and Clinical Data

arXiv:2609.12410v1 Announce Type: new Abstract: Dementia is a major and growing global health burden, with Alzheimer's disease (AD) accounting for most cases. Timely and accurate diagnosis is central

Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision

arXiv:2504.04903v3 Announce Type: replace Abstract: We present Lunima-OmniLV (abbreviated as OmniLV), a universal multimodal multi-task framework for low-level vision that addresses over 100 sub-tasks

MARE: Multimodal Alignment and Reinforcement for Explainable Deepfake Detection via Vision-Language Models

arXiv:2601.20433v4 Announce Type: replace Abstract: Deepfake detection is a widely researched topic that is crucial for combating the spread of malicious content, with existing methods mainly modeling

safetyarxiv-cs-cv

Multimodal Floorplan Encoding: Learning Dense Modality-Invariant Representations

arXiv:2609.12723v1 Announce Type: new Abstract: Floorplans arise in many forms, from vector CAD drawings to raster renderings and sensor-derived density maps. This heterogeneity makes it difficult to

safetyarxiv-cs-cv

Bidirectional Multimodal Fusion of Sky Images and Time-Series for Solar Forecasting with Large Language Models

arXiv:2609.11135v1 Announce Type: new Abstract: Short-term photovoltaic (PV) power and global horizontal irradiance (GHI) forecasts are essential for effective dispatch, reserve scheduling, and grid o

local-aiarxiv-cs-lg

BodyCam-VQA: Enhanced Body-Worn Camera Video Captioning via Multimodal Reasoning and Probe Question Generation

arXiv:2609.10815v1 Announce Type: cross Abstract: Police body-worn camera (BWC) footage has emerged as a critical aspect of law enforcement that ensures legal transparency, officer accountability, and

OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models

model-releasesarxiv-cs-cl

arXiv:2609.11244v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) have achieved remarkable progress across diverse tasks, they suffer from hallucinations where generated o

Understanding In-Context Multimodal Jailbreaks via Posterior Reweighting

arXiv:2609.10613v1 Announce Type: cross Abstract: In-context learning (ICL) jailbreaks reveal a critical vulnerability in multimodal large language models (MLLMs): harmful demonstrations in the prompt

safetyarxiv-cs-lg

Multimodal Emotion Recognition in Conversations via Class-Wise Adaptive Modality Fusion and Affective Geometry

arXiv:2609.09924v1 Announce Type: new Abstract: Emotion Recognition in Conversations (ERC) requires integrating heterogeneous textual, audio, and visual cues while accounting for conversational contex

10 Sept 2026

Scratchy: Visual-Scratchpad Multimodal Reasoning for Cryptographic Proof Generation in EasyCrypt

model-releasesarxiv-cs-ai

arXiv:2609.06226v1 Announce Type: cross Abstract: Large language models (LLMs) have recently made substantial progress in formal proof generation, yet presenting distinctive challenges in cryptographi

10 Sept 2026

Tutorials

A Specialized Large Multimodal Model for Interpreting PET/CT in Head and Neck Cancer

arXiv:2609.05532v1 Announce Type: new Abstract: Background: Diagnosing head and neck cancer using PET/CT is clinically challenging and time-consuming due to the anatomical complexity of the region, mo

tutorialsarxiv-cs-cv

AD-FM: Multimodal LLMs for Anomaly Detection via Multi-Stage Reasoning and Fine-Grained Reward Optimization

arXiv:2508.04175v2 Announce Type: replace Abstract: While Multimodal Large Language Models (MLLMs) demonstrate remarkable capabilities across diverse domains, their application to specialized anomaly

local-aiarxiv-cs-cv

CAVEAT: Recurrent Multimodal Diffusion Planning for Mapless Aerial Exploration

arXiv:2609.06623v1 Announce Type: cross Abstract: Can exploratory UAV waypoint sequences be generated from multimodal onboard observations and a fixed-dimensional recurrent internal state without main

local-aiarxiv-cs-lg

Agents

CUSP: Decomposable Collective Uncertainty for Multi-Agent Multimodal Reasoning

arXiv:2609.05708v1 Announce Type: cross Abstract: Aggregating heterogeneous vision-language models (VLMs) can improve multimodal reasoning, but neither an individual model's confidence nor that of the

agentsarxiv-cs-lg

From Coordinates to Candidate Regions: Temporal Change Localization via Region Selection in Remote Sensing Multimodal LLMs

arXiv:2609.08391v1 Announce Type: new Abstract: Remote sensing multimodal large language models (RS-MLLMs) have advanced scene understanding and visual question answering over satellite imagery, yet l

local-aiarxiv-cs-cv

InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling

arXiv:2609.07874v1 Announce Type: new Abstract: Multimodal large language models often capture visual-linguistic correlations but struggle to predict how local visual interventions propagate and affec

local-aiarxiv-cs-lg

Layer-Wise Gate-Controlled Prompt Truncation in a Multimodal Chest X-Ray Classifier

arXiv:2609.06590v1 Announce Type: cross Abstract: Mixture of Prompt Experts (MoPE) adapts multimodal transformers through input-dependent prompt composition, while retaining a fixed prompt length. We

Vision-Guided Text Prompt Tuning for Multimodal Sentiment Analysis

arXiv:2609.06497v1 Announce Type: cross Abstract: Multimodal sentiment analysis requires effective modeling of both verbal semantics and non-verbal affective cues. A central challenge is to calibrate

Latent-Aligned Reasoning for Multimodal Recommendation

arXiv:2609.04645v1 Announce Type: cross Abstract: Multimodal Vision-Language Models (VLMs) have demonstrated remarkable capabilities in cross-modal understanding, yet a fundamental challenge persists

7 Sept 2026

WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing

arXiv:2609.05171v1 Announce Type: new Abstract: Image generation and editing models have advanced rapidly, yet remain unreliable when prompts require external world knowledge. Bounded and long-tail pa

7 Sept 2026

Tutorials

Deploy a multimodal WhatsApp ordering assistant with Amazon Bedrock AgentCore

Learn how to deploy a multimodal WhatsApp ordering assistant that takes customer orders through text, voice notes, and real-time voice calls on a single business number, built on Amazon Bedrock AgentC

tutorialsaws-ml-blog

4 Sept 2026

Puffin-World: Scaling a Unified Multimodal Model with Native 3D World States

arXiv:2609.04196v1 Announce Type: new Abstract: We propose Puffin-World, a unified multimodal architecture that integrates physical understanding, spatial simulation, and 3D world generation and recon

4 Sept 2026

A Data-Driven Multimodal Method for Early Detection of Coordinated Abnormal Behaviors in Live-Streaming Platforms

arXiv:2609.01649v1 Announce Type: cross Abstract: With the rapid growth of live-streaming e-commerce and digital marketing, abnormal marketing behaviors have become increasingly concealed and coordina

safetyarxiv-cs-ai

3 Sept 2026

Beyond Appearance: Can Multimodal Large Language Models Exploit Vertical Structure for Remote Sensing Natural Scene Understanding?

arXiv:2605.25784v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have advanced rapidly in remote-sensing analysis, yet existing evaluations remain predominantly 2D-centric.

3 Sept 2026

MARS: What Retrieval Signals Are Hidden in Multimodal Large Language Models for Text-Video Retrieval?

arXiv:2609.02565v1 Announce Type: new Abstract: Text-video retrieval requires representations that can distinguish videos with similar scenes, actions, and temporal patterns. Recent multimodal large l

3 Sept 2026

Beyond Language Priors: Diagnosing and Fixing Visual-Origin Hallucinations in Multimodal LLM

arXiv:2609.00231v1 Announce Type: new Abstract: Existing research on object hallucination in multimodal large language models (MLLMs) predominantly attributes the problem to language priors such as ov

2 Sept 2026

Tutorials

CGFM-Nav: Cognitive Graph-Field Memory for Semantic-Guided Lifelong Multimodal Embodied Navigation

arXiv:2608.29114v1 Announce Type: cross Abstract: Vision-and-Language Navigation (VLN) requires agents to reason over accumulated observations while continuously exploring unseen regions. However, exi

tutorialsarxiv-cs-ai

Looking Again: Measuring Sycophancy in the Reasoning Chains of Multimodal Models Under Pressure

model-releasesarxiv-cs-ai

arXiv:2608.28623v1 Announce Type: cross Abstract: Large multimodal reasoning models (LMRMs) are getting increasingly capable, primarily through generating explicit chain-of-thought reasoning before an

Multimodal Feature Prototype Learning for Interpretable and Discriminative Cancer Survival Prediction

arXiv:2510.06113v2 Announce Type: replace Abstract: Survival analysis plays a vital role in making clinical decisions. However, the models currently in use are often difficult to interpret, which redu

local-aiarxiv-cs-cv

Toward Postural State Classification in Immersive VR with Multimodal Data and Explainability Analysis

arXiv:2608.28844v1 Announce Type: cross Abstract: Ensuring a safe virtual reality (VR) experience requires systems that can predict and respond when users lose their balance. Although prior work has e

safetyarxiv-cs-ai

Uncertainty-Aware Multimodal Anti-UAV Detection via Evidential Fusion and Conflict-Discounted Belief Aggregation

arXiv:2608.29235v1 Announce Type: new Abstract: Anti-UAV perception systems must remain reliable when sensor streams degrade under occlusion, fast motion, or modality-specific failure. Existing multim