Search: “multimodal”

EngIntervene: Benchmarking Multimodal Engineering State Understanding and Design Intervention Reasoning

arXiv:2609.33261v1 Announce Type: new Abstract: Multimodal engineering benchmarks largely evaluate static understanding, such as recognizing components, interpreting diagrams, or answering technical q

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding

DGX agent

arXiv:2609.34672v1 Announce Type: new Abstract: Fine-grained visual understanding requires models to recognize small details within complex images. Multimodal on-policy self-distillation (OPSD) addres

safetyarxiv-cs-cv

29 Sept 2026

Learning Multimodal Embeddings with Evidence-Aligned Readout

arXiv:2609.33659v1 Announce Type: new Abstract: Multimodal large language models can expose task-relevant evidence through generation, but producing useful evidence does not by itself determine how it

OmniSmartHome: A Multimodal Reasoning Benchmark for Smart-Home Agents

arXiv:2609.32569v1 Announce Type: new Abstract: Smart-home assistants are expected to handle diverse, realistic requests that arise in daily life. In such interactions, users often rely on the surroun

ORAV: Benchmarking Audio-Video Generation from Multimodal Contexts

arXiv:2609.34843v1 Announce Type: new Abstract: Audio-video generation using heterogeneous multimodal references has emerged as a new challenge, requiring both compositional control over generation an

PACGNet: Pyramidal Adaptive Cross-Gating Network for Multimodal Object Detection in Aerial Imagery

arXiv:2512.18291v3 Announce Type: replace Abstract: Object detection in aerial imagery is challenging because targets are often small, backgrounds are cluttered, and imaging conditions vary substantia

Agents

Retrieved but Not Delivered: Multimodal Memory Delivery for Long-Term Agents

arXiv:2609.32590v1 Announce Type: new Abstract: Work on memory for multimodal agents optimizes what is written, updated and retrieved. Between retrieval and the answer, however, is a stage that multim

agentsarxiv-cs-cv

SafeMol: Dual-Modality Safety Alignment for Molecular Multimodal Models

model-releasesarxiv-cs-lg

arXiv:2609.33640v1 Announce Type: new Abstract: Molecular multimodal models support diverse understanding and generation tasks but may introduce safety vulnerabilities when handling hazardous molecule

Source-Lifted Flow Matching for Intervenable Multimodal Imitation

model-releasesarxiv-cs-ro

arXiv:2607.10206v2 Announce Type: replace Abstract: Flow-matching policies are promising for imitation learning because they model complex multimodal action distributions. However, their stochasticity

The Plot Twist: Jailbreaking Unified Multimodal Models with a Three-Act NarrativeAttack

arXiv:2509.26473v2 Announce Type: replace Abstract: Unified Multimodal Understanding and Generation Models (UMMs) increasingly combine visual understanding and image generation within a single interac

28 Sept 2026

ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs

arXiv:2609.31448v1 Announce Type: cross Abstract: Clinical prediction models estimate risk from patient measurements, while large language models support medical text understanding and question answer

28 Sept 2026

AgenticCADedit: A Stateful, Tool-Mediated Agentic Approach to Multimodal 3D CAD Editing

arXiv:2609.29621v1 Announce Type: new Abstract: Computer-aided design is central to industrial manufacturing, and much of a designer's daily work consists of editing existing models from multimodal re

25 Sept 2026

ICE: Task-Aligned Clifford Latent Fields for Multimodal Graph Foundation Models

arXiv:2609.29398v1 Announce Type: new Abstract: Multimodal attributed graphs connect entities, visual content, language, and observed relations. Learning one foundation across such graphs requires mor

researcharxiv-cs-lg

25 Sept 2026

UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm

arXiv:2609.27915v1 Announce Type: new Abstract: Despite remarkable advancements in multimodal large language models (MLLMs), their fine-grained visual understanding is constrained by a primary relianc

24 Sept 2026

VCMM: Variance-Calibrated Momentum for Multimodal Learning

arXiv:2609.27577v1 Announce Type: new Abstract: Multimodal joint training often suffers from modality imbalance, where a dominant modality suppresses the optimization of others. Existing methods mainl

safetyarxiv-cs-lg

24 Sept 2026

Local Ai

BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception

arXiv:2609.25891v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) often struggle with fine-grained visual perception when processing complete images, as critical evidence may on

local-aiarxiv-cs-cv

From Research Frontier to Laboratory Bench: Design of a Four-Tier Experimental Teaching System for Multimodal Medical Image Intelligent Diagnosis

arXiv:2609.22790v1 Announce Type: new Abstract: Undergraduate programmes in intelligent medical engineering are expanding, yet laboratory curricula lag behind the multimodal, long-tailed, and distribu

safetyarxiv-cs-ai

Agents

ICDAR2026 Competition on Multimodal Reasoning over Documents in Multiple Domains

arXiv:2609.25055v1 Announce Type: new Abstract: In this report we present results of the ICDAR2026 Competition on Multimodal Reasoning over Documents in Multiple Domains. This competition aimed to adv

agentsarxiv-cs-cl

Learning to Plan in Human-Robot Collaboration: Multimodal Reinforcement Learning for Adaptive Interaction

arXiv:2609.25274v1 Announce Type: new Abstract: Robot assistants for older adults and people with disabilities need to perform collaborative tasks with users effectively. The core component of these s

safetyarxiv-cs-ro

MatchFusion: Explicit-Implicit Instance Matching for Spatio-Temporal Multimodal Autonomous Driving

arXiv:2609.25860v1 Announce Type: cross Abstract: Sparse instance representations provide a compact interface for spatial LiDAR-camera and temporal past-current interaction in multimodal perception an

safetyarxiv-cs-ro

Virtual Encoders in Multimodal Transformers

arXiv:2609.26513v1 Announce Type: new Abstract: Multimodal language models traditionally rely on dedicated perceptual encoders to construct task-usable representations. More integrated architectures h

Dynamic Thermal Gaussians: Multimodal 4D Gaussian Splatting

arXiv:2609.24531v1 Announce Type: new Abstract: Thermography plays a vital role in military and broader thermal analysis applications. Recent progress in 3D thermal reconstruction has extended tempera

GeoBalance: Geometry-Aware Monitoring and Reconstruction with Asymmetric Optimization for Balanced Multimodal Learning

arXiv:2609.23533v1 Announce Type: new Abstract: Multimodal classifiers can converge to modality-dominant solutions in which one modality dominates the joint prediction, suppressing the learning of oth

Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models

arXiv:2609.23715v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) incur substantial computational overhead due to the reliance on hundreds of visual tokens to represent images.

MIGU: Multimodal Instruction Grounding under Uncertainty for Manipulation Planning

model-releasesarxiv-cs-ro

arXiv:2609.24995v1 Announce Type: new Abstract: Understanding natural human instructions is crucial for deploying robots in human-centric environments. We study multimodal instruction grounding, where

Agents

MM-ContextFold: Context Folding for Multimodal Agentic Retrieval

arXiv:2609.23121v1 Announce Type: new Abstract: Multimodal Agentic Retrieval (MAR) requires agents to solve complex information-seeking tasks by iteratively invoking external tools. Typical frameworks

agentsarxiv-cs-cv

Summarize, Judge, Refine: Decoupled Content Understanding and Policy Learning for Multimodal Content Moderation

arXiv:2609.22094v1 Announce Type: new Abstract: Content moderation systems traditionally entangle multimodal understanding with policy-specific classification, requiring full pipeline retraining for e

safetyarxiv-cs-cl

DRT: Dense Reasoning Trace for Efficient and Grounded Multimodal Reasoning

arXiv:2609.21675v1 Announce Type: new Abstract: Despite the remarkable progress in Multimodal Large Language Models (MLLMs), prevailing Chain-of-Thought (CoT) paradigms remain confined to the natural-

21 Sept 2026

M2G-LLM: Enhancing Clinical Prediction via Multimodal Graph Reasoning and LLM Context Injection

arXiv:2609.21164v1 Announce Type: new Abstract: Integrating diverse data modalities --- such as clinical notes, laboratory results, and medical imaging --- is essential for advancing clinical decision

safetyarxiv-cs-lg

21 Sept 2026

PolyBridgeBench: Benchmarking Multimodal LLMs for Physics-Grounded Bridge Design

arXiv:2609.21493v1 Announce Type: new Abstract: Multimodal large language models, or MLLMs, perform well at visual understanding and structured generation, yet these capabilities do not establish whet

21 Sept 2026

Applications

Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts

arXiv:2609.18385v1 Announce Type: new Abstract: Emotions are an essential aspect of human communication, particularly on social media, where authors frequently combine text and images to convey their

applicationsarxiv-cs-cl

17 Sept 2026

Lost in Perception: Isolating Perceptual and Reasoning Failures in Multimodal Physics and Geometry Reasoning

arXiv:2609.18991v1 Announce Type: new Abstract: Multimodal LLMs report strong performance on scientific reasoning benchmarks, yet most treat perception and reasoning as a single measurable process. We

17 Sept 2026

Multimodal Conditioning of Fine-Tuned Stable Diffusion XL for Controllable and Culturally Faithful Ulos Motif Generation

arXiv:2609.17987v1 Announce Type: new Abstract: The traditional Batak Ulos weaving industry faces growing challenges in producing diverse, innovative motifs due to limitations in conventional, manuall

17 Sept 2026

Multimodal Cultural Heritage Architectural Style Classification for Residential Buildings in the UAE Based on CLIP Embeddings and SVM

arXiv:2609.17181v1 Announce Type: cross Abstract: The analysis and classification of cultural heritage architectural styles remain challenging due to the complexity of visual images of buildings, whic

16 Sept 2026

Negation Beyond the Verbal Channel: Temporal Multimodal Correlates in Dialogue

arXiv:2609.16396v1 Announce Type: new Abstract: Negation is typically modeled through its linguistic realization, although spoken interaction is accompanied by tightly coordinated nonverbal behavior.

16 Sept 2026

Unsafe by Reciprocity: How Generation-Understanding Coupling Undermines Safety in Unified Multimodal Models

arXiv:2603.27332v2 Announce Type: replace Abstract: Recent advances in Large Language Models (LLMs) and Text-to-Image (T2I) models have led to the emergence of Unified Multimodal Models (UMMs), where

safetyarxiv-cs-cv

16 Sept 2026

A Comprehensive Review of Multimodal Facial State Analysis: Tasks, Methods, and Resources

arXiv:2609.13255v1 Announce Type: new Abstract: Facial state analysis plays a crucial role in understanding human expressions, psychological modeling, and human computer interaction. Traditional unimo

A Language-Guided Multimodal Foundation Model for Zero-Shot and Multi-Task Brain Signal Analysis

arXiv:2609.15740v1 Announce Type: cross Abstract: Brain signal analysis is essential for both neuroscience research and clinical diagnostics, yet current approaches face critical limitations. End-to-e

safetyarxiv-cs-ai

ArtSociety: Multi-Agent Multimodal Collaboration for Art Emotion Understanding

arXiv:2609.13240v1 Announce Type: new Abstract: The AffectiveArt Multidimensional Art Emotion Understanding task asks to jointly predict an artwork's fine-grained emotion (12 classes, 1549:1 head-to-t

safetyarxiv-cs-cv

AURA: Unified Multimodal Framework for Conversational Music Editing

arXiv:2609.14344v1 Announce Type: cross Abstract: Instruction-guided music editors typically process each request independently, limiting their ability to support workflows in which users progressivel

Exploring Multimodal Turn-Taking Cues in Face-to-Face Conversation using Voice Activity Projection

arXiv:2609.14666v1 Announce Type: cross Abstract: Turn-taking is a fundamental component of spoken interaction, and while humans naturally rely on both verbal and non-verbal signals, dialogue systems

Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations

arXiv:2410.15359v2 Announce Type: replace Abstract: Multimodal Foundation Models represent a significant leap in artificial intelligence. Among them, Large Vision-Language Models (LVLMs) serve as the

Modality-Guided Mixture of Structured Experts with Entropy-Triggered Routing for Multimodal Recommendation

arXiv:2602.20723v3 Announce Type: replace Abstract: Multimodal recommenders combine collaborative behavior with visual and textual item evidence, whose usefulness varies across user-item interactions.

Calibrated Ambiguity in Multimodal Language Models: Humans reach for cultural references, while models describe the picture

arXiv:2609.12575v1 Announce Type: new Abstract: Ambiguity is often treated as a bug for AI systems to resolve---but in human communication and culture, ambiguity can also be a generative resource. Fro

14 Sept 2026

FINESSE: An Agent-Based Simulator and Benchmark Dataset for Multimodal Financial Event Sequences

model-releasesarxiv-cs-lg

arXiv:2609.11993v1 Announce Type: new Abstract: Machine learning research in financial services is limited by the scarcity of representative open-source datasets. Existing resources are often narrowly

14 Sept 2026

QuPAINT: Physics-Aware Multimodal Reasoning for Quantum Material Characterization

arXiv:2609.12202v1 Announce Type: new Abstract: Characterizing two-dimensional (2D) quantum materials by optical microscopy requires localizing exfoliated flakes and determining their layer thickness

14 Sept 2026

MultiHuSE: A Multimodal Dataset for Humour Styles and Emotions

arXiv:2609.11322v1 Announce Type: new Abstract: Computational recognition of verbal humour remains a challenging task, requiring an understanding of language, delivery style, emotions, and cultural co

11 Sept 2026

RiVaT-Fuse: Reliability-Calibrated Variational Tensor Fusion for Multimodal Prediction under Modality Uncertainty

arXiv:2609.10798v1 Announce Type: cross Abstract: Image-metadata prediction requires fusing heterogeneous evidence whose reliability can vary across samples and latent factors. Existing representation