Search: “multimodal”

RSFusionDet: Underwater RGB-Sonar Multimodal Object Detection

arXiv:2608.25367v1 Announce Type: new Abstract: Underwater unimodal object detection faces many challenges in sensor imaging, such as optical images limited by underwater noise and visible distance, a

27 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

A Source-Grounded Framework for Constructing and Evaluating Progressive Multimodal Diagnostic Dialogues from Clinical Case Reports

DGX agent

arXiv:2608.22713v1 Announce Type: new Abstract: Clinical diagnosis requires progressive integration of patient history, physical examination, laboratory findings, medical images, and diagnostic-inform

model-releasesarxiv-cs-cl

25 Aug 2026

Dual-Grained Agent Memory and Shapley Context Attribution for Multimodal Agentic Learner

arXiv:2608.23268v1 Announce Type: new Abstract: Frontier multimodal large language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical reasoning. Parameter-lev

Let the Bullets Fly: Multimodal Fake News Detection with Temporal-Aligned Generative Danmaku

arXiv:2608.22832v1 Announce Type: new Abstract: The social interactions among crowds via extit{Danmaku} (a.k.a., bullet comments) on modern multimedia platforms can facilitate both viewpoint conflicts

MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesis

arXiv:2608.22323v1 Announce Type: new Abstract: The application of Large Language Models (LLMs) to diagnostic decision-making has garnered growing interest. However, existing benchmarks largely focus

Multimodal Injury Risk and Performance Prediction in Tennis Using Weighted Ensemble Learning

arXiv:2608.21530v1 Announce Type: new Abstract: Machine learning has had a positive impact on the sports industry, with one of its most promising applications being the prediction of athlete performan

researcharxiv-cs-lg

Multimodal pseudo-CT synthesis for PET attenuation correction using separate modality encoding and topogram conditioning

arXiv:2608.21481v1 Announce Type: cross Abstract: We participated in the BIC-MAC Challenge with a multimodal 3D patch-based U-Net for pseudo-CT generation from NAC-PET, MRI, and 2D topograms. By using

Query-Driven Multimodal Information Extraction from Long Documents

arXiv:2608.22214v1 Announce Type: new Abstract: In domain-specific multimodal long documents, images and text jointly convey complex knowledge that cannot be fully captured by plain text alone. Howeve

Reliability- and Anatomy-Consistency-Aware Multimodal Learning for Robust Fracture Classification from Bangladeshi Radiographs

arXiv:2608.21482v1 Announce Type: cross Abstract: Background: Multimodal fracture classifiers may benefit from patient and anatomical metadata, but they can also become brittle when contextual informa

Tutorials

When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?

arXiv:2608.22174v1 Announce Type: new Abstract: Unified multimodal models (UMMs) can perform both understanding and generation, raising a central question: can visual generation improve understanding?

tutorialsarxiv-cs-cv

COMET: Contrastive Motion-Enhanced Temporal Reasoning for Video Multimodal Large Language Models

arXiv:2608.21030v1 Announce Type: cross Abstract: Video multimodal large language models have advanced significantly, yet fine-grained motion-temporal understanding remains fragile. The core bottlenec

safetyarxiv-cs-cl

24 Aug 2026

Beyond Multimodal Alignment: Certifying Physical Language through Response Substitution and Ordered Execution

arXiv:2608.19492v1 Announce Type: new Abstract: World models increasingly treat compact multimodal representations as interfaces between perception and physical interaction, yet existing probes do not

safetyarxiv-cs-lg

21 Aug 2026

Reinforcing Egocentric Spatial Perception in Multimodal Large Language Models via Ego Scene Augmentation

arXiv:2607.14497v2 Announce Type: replace Abstract: Egocentric Visual Question Answering (VQA) has attracted widespread attention as an important task for enabling Multimodal Large Language Models (ML

21 Aug 2026

Time-Series Retrieval for Grounding Multimodal Language Models in Remaining Useful Life

arXiv:2608.19218v1 Announce Type: cross Abstract: Large language models (LLMs) and agentic AI systems are increasingly being explored for domain-specific maintenance and prognostics tasks, raising the

21 Aug 2026

Million-scale multimodal pollen microscopy with expert-guided foundation models

arXiv:2606.17809v2 Announce Type: replace Abstract: Automated pollen identification from microscopy remains a bottleneck in aerobiology, palaeoecology and biodiversity monitoring, because scalable sys

20 Aug 2026

When Two Tracers Disagree: An Investigation of Multimodal Fusion for Clinical PET/CT Segmentation

arXiv:2608.19063v1 Announce Type: new Abstract: PSMA and FDG PET/CT visualise complementary biological information in prostate cancer. Combining both tracers could capture heterogeneous tumour phenoty

20 Aug 2026

Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Models

model-releasesarxiv-cs-cl

arXiv:2608.17102v1 Announce Type: new Abstract: Modern multimodal foundation models (MFMs) have made rapid progress on tasks requiring integrated perception across speech, vision, and language, includ

19 Aug 2026

PRISM: Precision and contact-rich Real-world Industrial Skill dataset with Multimodal sensing

model-releasesarxiv-cs-ro

arXiv:2608.17962v1 Announce Type: new Abstract: Recent progress in robotic learning has been fueled by large-scale datasets collected in everyday environments. However, most existing datasets emphasiz

19 Aug 2026

Local Ai

EgoGazeLite: On-Device Egocentric Gaze Prediction for Token-Efficient Multimodal LLM Video Input

arXiv:2608.15614v1 Announce Type: cross Abstract: The use of multimodal LLMs (MLLMs) for egocentric video understanding with wearable devices is constrained by the token budget. Memory and compute cos

local-aiarxiv-cs-ai

Local Ai

GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning

arXiv:2608.15517v1 Announce Type: new Abstract: Chain-of-thought reasoning has substantially improved the problem-solving capabilities of multimodal large language models. Fine-grained visual evidence

local-aiarxiv-cs-cv

MetaReason: Precise Interleaved Multimodal Reasoning via Editing Meta Information for Solving Geometry Problems

arXiv:2608.15006v1 Announce Type: cross Abstract: Although visual reasoning is crucial for solving complex geometry tasks, existing vision-language models rely heavily on text-only reasoning. Some rec

MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarios

arXiv:2505.21333v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved considerable accuracy in Optical Character Recognition (OCR) from static images. However, the

Applications

PolyDebate: A Game-Orchestrated Multimodal System for Debate Skills Practice and Evaluation

arXiv:2608.16276v1 Announce Type: cross Abstract: Debate is a structured form of persuasive communication that trains argument construction, rebuttal, oral delivery, and audience awareness. These skil

applicationsarxiv-cs-cl

UrbanWorld2.0: A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scale

arXiv:2511.18005v2 Announce Type: replace Abstract: The automated generation of high-fidelity, city-scale 3D environments remains a formidable challenge with profound academic and industrial implicati

safetyarxiv-cs-cv

Local Ai

Zero-MELO: Test-Time Evidence Calibration with Multimodal LLMs for Zero-Shot Micro-Gesture Recognition

arXiv:2608.14854v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) excel in general video understanding, their capability in fine-grained and motion-centric tasks remains l

local-aiarxiv-cs-cv

GBU-Palm: A Multimodal Video Dataset and Benchmark for Palm Presentation Attack Detection

arXiv:2608.14389v1 Announce Type: cross Abstract: Existing palm presentation attack detection (PAD) datasets are often limited by static imagery, restricted acquisition conditions, or insufficient mul

17 Aug 2026

Label-Free Deep-Tissue Peripheral Nerve Detection with a Handheld Multimodal OCT Probe and NerveDetNet

arXiv:2608.13807v1 Announce Type: cross Abstract: Peripheral nerves buried beneath intact tissue are difficult to visualize during surgery and remain inaccessible to white light wide-field imaging and

17 Aug 2026

Attribute-Conditioned Multimodal Slot Factorization for Controllable Fashion Retrieval

arXiv:2608.12570v1 Announce Type: new Abstract: Fashion retrieval often requires satisfying multiple attributes at once, such as category, color, pattern, and demographic. Monolithic embeddings mix th

Hardware

OpenRC: An Open-Source Robotic Colonoscopy Framework for Multimodal Data Acquisition and Autonomy Research

arXiv:2604.03781v2 Announce Type: replace Abstract: Colorectal cancer screening critically depends on colonoscopy, yet existing platforms offer limited support for systematically studying the coupled

hardwarearxiv-cs-ro

Semantic Steering for Controllable Generation: Tuning-Free Concept Erasure in Multimodal Diffusion Transformers

arXiv:2608.12829v1 Announce Type: new Abstract: Multimodal Diffusion Transformers (MM-DiTs) have demonstrated remarkable text-to-image generation performance, surpassing traditional U-Net-based diffus

safetyarxiv-cs-cv

The Role of Natural Language Understanding in Multimodal Video-Based Dengue Diagnosis

arXiv:2608.12677v1 Announce Type: new Abstract: Detecting infection-related behavioral changes in mosquitoes from video data is challenging because mosquitoes are small, move rapidly and irregularly,

Motion-as-Prompt: Enhancing Motion Reasoning in Multimodal Large Language Models via Motion-Guided Cross-Frame Visual Prompting

arXiv:2608.11655v1 Announce Type: cross Abstract: Motion-centric video reasoning is fundamental to interactive applications such as robotic manipulation and autonomous navigation. However, multimodal

13 Aug 2026

Applications

MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation

arXiv:2606.17598v2 Announce Type: replace-cross Abstract: Humans naturally leverage diverse sensing modalities to interact with the physical world, while most Vision-Language-Action (VLA) models for r

applicationsarxiv-cs-cv

13 Aug 2026

Multimodal Item Parameter Estimation using Simulated Response Probabilitie

arXiv:2608.10154v1 Announce Type: cross Abstract: We present results from reconstructing multiple-choice model (MCM) and three-parameter logistic (3PL) model curves using a fine-tuned multimodal large

12 Aug 2026

EMMR: Emotion-Mediated Multimodal Reasoning for Personality Assessment in Asynchronous Video Interviews

arXiv:2608.07512v1 Announce Type: cross Abstract: Asynchronous Video Interviews (AVIs) have become increasingly popular for personality assessment. Recent large language models (LLMs) have shown poten

Hardware

EsaacSim: A Multimodal Event Camera Add-on for NVIDIA Isaac Sim

arXiv:2608.08522v1 Announce Type: new Abstract: Event-based vision is becoming an increasingly important sensing paradigm for robotics, yet its adoption remains limited by sensor availability and the

hardwarearxiv-cs-ro

FitAQA: A Benchmark of Fitness Action Quality Assessment for Multimodal Large Language Models

arXiv:2608.08736v1 Announce Type: new Abstract: Fitness Action Quality Assessment (AQA) is important for intelligent sports training, yet the capabilities of Multimodal Large Language Models (MLLMs) i

Improving Generalization Robustness of Multimodal RLVR

arXiv:2608.08802v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) makes Multimodal Large Language Models more accurate, but the gains are brittle: simply paraphrasi

Tutorials

Let Geometry GUIDE: Layer-wise Unrolling of Geometric Priors in Multimodal LLMs

arXiv:2604.05695v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress in 2D visual tasks but still struggle to understand physical space in rea

tutorialsarxiv-cs-cv

MMArch: Benchmarking Multimodal Reasoning Grounded in Architectural Evidence

arXiv:2608.09281v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) perform strongly on engineering imagery, yet existing benchmarks mostly test drawing recognition, information e

TGIF: Text-Guided Layer Fusion Mitigates Hallucination in Multimodal LLMs

arXiv:2601.03100v3 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) typically rely on a single late-layer feature from a frozen vision encoder, leaving the encoder's ric

TomaMMU: A Comprehensive Multimodal Understanding Benchmark for Tomato Leaf Diseases

arXiv:2608.08727v1 Announce Type: cross Abstract: To address this gap, we introduce TomaMMU, a large-scale Tomato leaf disease MultiModal Understanding dataset, alongside TomaBench, a benchmark for ev

VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs

arXiv:2510.16598v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) encounter significant computational and memory bottlenecks from the massive number of visual tokens generat

Deep Evidential Regression for Sparse Forest Height Estimation from Multimodal Satellite Imagery

arXiv:2608.06406v1 Announce Type: new Abstract: Accurate estimation of forest height from satellite imagery is essential for applications such as carbon accounting, biodiversity monitoring, and ecosys

10 Aug 2026

M^3R-Bench: A Unified Benchmark for Evidence-Grounded Multimodal Metaphor Understanding

model-releasesarxiv-cs-cl

arXiv:2608.05817v1 Announce Type: new Abstract: Metaphor enables the understanding of abstract concepts through cross-domain mappings while conveying affective attitudes. In multimodal scenarios, visu

7 Aug 2026

MM-ISTS: Cooperating Irregularly Sampled Time Series Forecasting with Multimodal Vision-Text LLMs

arXiv:2603.05997v2 Announce Type: replace-cross Abstract: Irregularly sampled time series (ISTS) are widespread in real-world scenarios, exhibiting asynchronous observations on uneven time intervals a

7 Aug 2026

PhaseCoder: Microphone Geometry-Agnostic Spatial Audio Understanding for Multimodal LLMs

arXiv:2601.21124v2 Announce Type: replace-cross Abstract: Current multimodal LLMs process audio as a mono stream, ignoring the rich spatial information essential for embodied AI. Existing spatial audi

7 Aug 2026

AI-driven Multimodal Representation Learning for Latent Mediation Structure Discovery of Socioeconomic Disadvantage, Psychosocial Factors, and Cardiometabolic Multimorbidity: Insights from the All of Us Research Program

arXiv:2608.04016v1 Announce Type: cross Abstract: Social disadvantage is associated with multimorbidity, but the pathways linking social conditions to disease burden remain poorly understood. We devel