Search: “multimodal”

RiVaT-Fuse: Reliability-Calibrated Variational Tensor Fusion for Multimodal Prediction under Modality Uncertainty

arXiv:2609.10798v1 Announce Type: cross Abstract: Image-metadata prediction requires fusing heterogeneous evidence whose reliability can vary across samples and latent factors. Existing representation

11 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Robust Multimodal Sentiment Analysis with Incomplete Modalities via Semantic-aware Completeness based Reconstruction

DGX agent

arXiv:2609.10950v1 Announce Type: new Abstract: Recent multimodal sentiment analysis studies increasingly adopt text-centric fusion approaches to exploit the rich sentiment information inherent in the

model-releasesarxiv-cs-cl

11 Sept 2026

Toward Interpretable Multimodal Fusion: Heat Conduction Modeling for Hyperspectral and LiDAR Joint Classification

arXiv:2609.11040v1 Announce Type: new Abstract: The fusion of hyperspectral (HS) and Light Detection and Ranging (LiDAR) data plays a crucial role in enhancing land-cover classification by jointly exp

11 Sept 2026

A Confidence-Aware Multimodal Fusion Framework for Industrial Human-Robot Collaboration

arXiv:2609.10339v1 Announce Type: new Abstract: A confidence-aware multimodal fusion framework (CAMF) is proposed to realize reliable human intention prediction for industrial human-robot collaboratio

safetyarxiv-cs-ro

10 Sept 2026

Eliciting Self-Verification in Multimodal Reasoning Agents with Reinforcement Learning

arXiv:2609.08025v1 Announce Type: cross Abstract: Reasoning agents increasingly rely on external tools such as web search to answer complex queries. Reinforcement learning (RL) finetuning algorithms s

EntropyPrune: Matrix Entropy Guided Visual Token Pruning for Multimodal Large Language Models

arXiv:2602.17196v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) incur substantial inference cost due to the processing of hundreds of visual tokens per image. Although tok

MoEMB: Scaling Universal Multimodal Embeddings with Efficient Mixture-of-Experts Models

arXiv:2609.08663v1 Announce Type: new Abstract: Universal multimodal embedding (UME) increasingly demands encoder's capacity for handling a broad range of tasks and modalities with increased complexit

Tutorials

TeMo: Temperature Modulation for Multimodal Contrastive Learning

arXiv:2609.07540v1 Announce Type: new Abstract: Contrastive learning approaches achieve strong performance by training models to bring similar samples closer while pushing dissimilar samples apart. A

tutorialsarxiv-cs-cv

Hardware

When to Use Encode-Prefill-Decode Disaggregation to Accelerate Multimodal Model Serving

Encode‑Prefill‑Decode (EPD) disaggregation is an inference optimization in NVIDIA Dynamo that separates vision encoding from LLM prefill–decode stages for multimodal requests. On image‑heavy prompts,

hardwarenvidia-developer

Cross-modal triage network: a multimodal deep learning framework for severity-based triage and visual explainability in chest radiographs

arXiv:2609.04357v1 Announce Type: cross Abstract: Purpose: Increased number of chest radiograph (CXR) scans create a triage bottleneck, queueing urgent examinations behind routine ones. Existing AI to

7 Sept 2026

MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression

arXiv:2609.04947v1 Announce Type: cross Abstract: Recently, multimodal large-scale reasoning models have demonstrated remarkable capabilities in solving complex tasks through long Chains-of-Thought (M

safetyarxiv-cs-ai

7 Sept 2026

Establishing a Dynamic Multimodal HRI Dataset for Engagement Analysis with a Humanoid Robot

arXiv:2609.03255v1 Announce Type: new Abstract: This paper presents an experimental design for constructing a multimodal dataset to analyze user engagement in human-robot interaction (HRI). Prior stud

researcharxiv-cs-ro

4 Sept 2026

Occlusion-Robust Multimodal Emotion Recognition in VR via Fusion of Facial Images and EMG

arXiv:2609.03569v1 Announce Type: new Abstract: Head-mounted displays (HMDs) fundamentally limit emotion recognition in virtual reality (VR): by occluding the upper face, they render conventional imag

4 Sept 2026

VisCAD: A Foundation Model Suite with Multimodal Industrial CAD Intelligence

arXiv:2609.03811v1 Announce Type: cross Abstract: AI-assisted computer-aided design (CAD) for industrial products involves two challenging phases. Part-level generation maps diverse forms of user inte

4 Sept 2026

Deeply Interleaved Text-Image Contexts for Multimodal LLMs Assessment

arXiv:2609.02573v1 Announce Type: new Abstract: Current evaluations and training of multimodal models predominantly focus on multi-image tasks, largely overlooking interleaved text-image scenarios. In

3 Sept 2026

Predictors of Loneliness in Older Adults Using Multimodal Analysis of Speech and Language

arXiv:2609.02606v1 Announce Type: new Abstract: Loneliness is a critical public health issue among older adults, linked to higher risks of depression, cognitive decline, and mortality. Scalable, objec

3 Sept 2026

TC-Next: Zero-Shot Multimodal Cyclone Forecasting

arXiv:2609.02085v1 Announce Type: cross Abstract: We present TropicalCycloneNext (TC-Next), a multimodal deep learning model that forecasts tropical cyclone track and intensity at 6-24 h leads by leve

3 Sept 2026

Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges

arXiv:2607.19011v2 Announce Type: replace-cross Abstract: Multimodal humor in memes, cartoons, and comics remains difficult for AI systems because intended meaning depends on non-literal mechanisms, s

Evaluating Multimodal LLMs as Generalist Vision-Language-Action Agents for Drone Control: Commanding, Approaching, Tracking and Searching

arXiv:2609.01404v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are strong perceivers of images and video. We ask how far that reach extends into acting: dropping an MLLM di

Local Ai

S^2Prune: Spatially Structured Visual Token Pruning for Multimodal Large Language Models

arXiv:2609.01224v1 Announce Type: new Abstract: Visual token pruning reduces the inference overhead of multimodal large language models (MLLMs) by retaining only a subset of visual tokens. Existing me

local-aiarxiv-cs-cv

SlideMix: Enhancing Whole Slide Image Analysis via Multimodal Shuffling

arXiv:2609.00396v1 Announce Type: new Abstract: Histopathological whole slide images (WSIs) are central to cancer diagnosis, but their gigapixel scale, tissue heterogeneity, weak slide-level supervisi

3D-MRL: Nested Multimodal 3D Representations via Matryoshka Representation Learning

arXiv:2608.29285v1 Announce Type: new Abstract: Vision-Language Models align point clouds with image and text embeddings, enabling zero-shot recognition, retrieval, and open-vocabulary understanding o

CM2: Multimodal Cultural Reasoning via an Integrated Multi-Agent Framework

arXiv:2608.30498v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have shown remarkable success in STEM domains, where progress is often driven by vertical, step-by-step deducti

agentsarxiv-cs-ai

Applications

Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation

arXiv:2608.30163v1 Announce Type: cross Abstract: Real-world knowledge resides in multimodal documents, necessitating retrieval-augmented generation (RAG) for accurate question answering. However, exi

applicationsarxiv-cs-cv

HalluPrism: When Multimodal Uncertainty Should Diagnose, Not Decide

arXiv:2608.29193v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) can assign similar confidence to answers that fail for different reasons. We propose HalluPrism, a behavioral d

Learning Where Outcomes Change:Credit-Addressable Reasoning for Multimodal Geometry

arXiv:2608.30457v1 Announce Type: cross Abstract: Multimodal geometry reasoning requires VLMs to extract precise visual relations and preserve them through multi-step deduction. Existing free-form tra

MMLongEmbed: Benchmarking Multimodal Embedding Models in Long-Context Scenarios

arXiv:2606.14747v2 Announce Type: replace-cross Abstract: Recent advancements have significantly expanded the theoretical context windows of Multimodal Embedding Models (MEMs). However, larger context

Read the Room, Read the Image: Understanding Indirect Speech Acts in Multimodal Visual Contexts

model-releasesarxiv-cs-cl

arXiv:2608.30270v1 Announce Type: new Abstract: Indirect speech acts (ISAs) require pragmatic reasoning over context, as directive intent can- not be inferred from surface form alone. Prior text-based

Toward Cultural Alignment: Human-Centered Evaluation of Multimodal AI Stories Across Five African Communities

arXiv:2608.29209v1 Announce Type: new Abstract: In this paper, we examine how well AI-generated multimodal stories align with the lived practices, relationships, language, values, and visual expectati

safetyarxiv-cs-cl

When History Is Multimodal: Rethinking Context Management for Long-Horizon Agents

arXiv:2608.29897v1 Announce Type: new Abstract: Long-horizon agents need a context manager to compress growing interaction histories into a bounded working context, via passive strategies or active st

safetyarxiv-cs-cl

Local Ai

GAAT: Geometry-Aware Alignment Transformer for Multimodal UAV Perception

arXiv:2608.27971v1 Announce Type: new Abstract: Unmanned aerial vehicle (UAV) multimodal perception integrates visible (RGB), infrared (IR), synthetic aperture radar (SAR), and depth sensors for scene

local-aiarxiv-cs-cv

31 Aug 2026

Semantic Head Specialization Guides Hybrid ViT Attention for Multimodal LLMs

model-releasesarxiv-cs-cl

arXiv:2608.28383v1 Announce Type: cross Abstract: Hybrid attention dominates frontier LLMs, yet Vision Transformers (ViTs) in multimodal LLMs lack a satisfactory hybrid design, with no consensus on wh

31 Aug 2026

GraphMemix: Query-Aware Evidence Forests for Long-Term Multimodal Agent Memory

arXiv:2608.26983v1 Announce Type: new Abstract: Organizing long-term memory for multimodal agents remains challenging because existing methods either suffer from expensive question-agnostic offline su

agentsarxiv-cs-ai

28 Aug 2026

CardioFusion-AI: Robust ECG--PPG Fusion for Multimodal Physiological Monitoring Under Signal Degradation

arXiv:2608.26000v1 Announce Type: cross Abstract: Wearable electrocardiogram (ECG) and photoplethysmogram (PPG) sensors are complementary but individually fragile: motion artifact, poor contact, and s

27 Aug 2026

Evidence-Grounded Mapping of Multimodal Human Sensing Psychological Transdiagnostic Dimensions

model-releasesarxiv-cs-lg

arXiv:2608.24903v1 Announce Type: cross Abstract: Mobile and wearable sensing enables longitudinal observation of behavior, yet translating these signals into meaningful mental health constructs remai

27 Aug 2026

Multimodal Injury Risk Prediction in Tennis

arXiv:2608.25126v1 Announce Type: new Abstract: Machine learning has had a significant positive impact on the prediction of athlete performance and injury risk. Most works in this field rely on subjec

27 Aug 2026

Industry

[1/7] 🚨 We're releasing LAION-BVD: a 10-million-hour open video dataset for multimodal pre-training. - 1.3B video URLs from CommonCrawl - 8…

[1/7] 🚨 We're releasing LAION-BVD: a 10-million-hour open video dataset for multimodal pre-training. - 1.3B video URLs from CommonCrawl - 80M downloaded videos - 10M video hours - 55M captioned clips

industryemad-mostaque--x

LG-GER: Language-Guided Group Emotion Recognition via Multimodal Evidence Distillation

arXiv:2608.23880v1 Announce Type: new Abstract: Inferring the collective emotional state of a group of people from a single image, a task known as group emotion recognition (GER), requires integrating

LUCAID: Agentic Multimodal AI for Lung Cancer Precision Pathology

arXiv:2608.23803v1 Announce Type: cross Abstract: Lung cancer tissue diagnostics is complex, as therapy decisions in precision oncology rely on the integration of histomorphological, immunohistochemic

agentsarxiv-cs-ai

TangramPuzzle: Evaluating Multimodal Large Language Models with Compositional Spatial Reasoning

arXiv:2601.16520v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress in visual recognition and semantic understanding, yet precise compo

The mystery model Ox Alpha is now available in @AutoClawAIer by @Zai_org It's official name is GLM-5.3-Flash, a next-generation multimodal m…

The mystery model Ox Alpha is now available in @AutoClawAIer by @Zai_org It's official name is GLM-5.3-Flash, a next-generation multimodal model for vision-language understanding, code generation, and

agentszhipu-ai--x

Beyond What Meets the Eye: Unveiling Situational Illusions for Multimodal Large Language Models

arXiv:2608.22232v1 Announce Type: new Abstract: Real-world situation appearances can deviate from their underlying physical states, challenging the reliability of multimodal large language models (MLL

CaRGo-T: Causal Reasoning Graph-of-Thought improves Multimodal Humor Comprehension

arXiv:2608.23172v1 Announce Type: new Abstract: Large-scale vision-language models (VLMs) have demonstrated remarkable versatility across a wide range of multimodal tasks. However, understanding humor

Hardware

CST: Collaborative Selective Transmission for Communication-Efficient Multimodal Edge Inference

arXiv:2608.22115v1 Announce Type: new Abstract: Collaborative multimodal inference improves edge perception by combining observations from distributed sensing devices, but transmitting high-dimensiona

hardwarearxiv-cs-lg

Evaluating Multimodal Narrative Understanding of Popular Hollywood Films

arXiv:2608.21430v1 Announce Type: new Abstract: Multimodal language models increasingly show promise for enabling the large-scale computational analysis of film, opening up new avenues for learning ab

Local Ai

HiMA-MDD: A Hierarchical Multi-Agent Harness for Interpretable Multimodal Depression Detection in Clinical Interviews

arXiv:2608.21868v1 Announce Type: new Abstract: Depression assessment from multimodal clinical interviews requires integrating dispersed evidence from multiple symptoms into a coherent PHQ-8 profile.

local-aiarxiv-cs-ai

Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression

arXiv:2606.14782v3 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) achieve strong vision-language reasoning but incur large KV caches and high decoding latency with lon

MCite-RL: Towards Reliable Multimodal RAG via Citation-enhanced Agentic Reinforcement Learning

arXiv:2608.21808v1 Announce Type: new Abstract: Multimodal Retrieval-Augmented Generation (RAG) with visual citation is crucial for ensuring the traceability and verifiability of MLLMs. However, curre

agentsarxiv-cs-cl