Search: “multimodal”

Artificial Intelligence Across the Cardiac Amyloidosis Diagnostic and Management Pathway: From Single-Modality Detection to Multimodal Clinical Integration

arXiv:2607.09948v2 Announce Type: replace-cross Abstract: Cardiac amyloidosis (CA) is increasingly recognized but remains substantially underdiagnosed because its clinical and imaging phenotype overla

researcharxiv-cs-lg

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

FOCUS: Benchmarking Retinal Model Generalization from Foundation Vision Encoders to Multimodal LLMs

DGX agent

arXiv:2609.33158v1 Announce Type: new Abstract: Progress in AI-based retinal image analysis has advanced with foundation models, yet evaluating their reliability remains challenging. Performance repor

model-releasesarxiv-cs-cv

29 Sept 2026

ForensicZoom: Adaptive Visual Inspection with Multimodal LLMs for Industrial-Grade Face Forgery Detection

arXiv:2609.31661v1 Announce Type: new Abstract: Reliable face forgery detection is critical to the security of online identity verification systems, where missed attacks compromise security and excess

Industry

How Condé Nast built multimodal video discovery with Amazon Bedrock

Condé Nast's editorial teams spent an average of 250 minutes per task searching a library of more than 140,000 videos using only titles and descriptions. Working with the AWS Generative AI Innovation

industryaws-ml-blog

Progressive-View On-Policy Distillation for Regional-to-Global Transfer in Multimodal LLMs

arXiv:2609.32333v1 Announce Type: new Abstract: Regional-to-global distillation uses crop-conditioned guidance to improve full-image understanding. The challenge is to effectively transfer the teacher

safetyarxiv-cs-cv

SetOPD: From Few Visual Exemplars to Multimodal Candidate Sets for Remote-Sensing Open-Prompt Detection

arXiv:2609.32529v1 Announce Type: new Abstract: Open-prompt detectors allow users to specify targets with text, visual exemplars, or both. We argue that existing designs underuse the visual modality i

THEIA: A Multimodal Dataset and Benchmark for Vision-Language Analysis of Layout

model-releasesarxiv-cs-lg

arXiv:2609.35035v1 Announce Type: new Abstract: The integration of artificial intelligence into computer-aided design frameworks has sparked a shift in the design of analog integrated circuits (ICs),

Tutorials

Weighting Schedules Govern What and When Score-Based Generative Models Learn from Multimodal Data

arXiv:2609.35322v1 Announce Type: new Abstract: Score-based generative models generate new samples by integrating a time-dependent drift that carries Gaussian noise onto the target distribution. In pr

tutorialsarxiv-cs-lg

Hardware

EAServe: Encode-Aware Disaggregated Serving for Multimodal Large Language Models

arXiv:2609.31551v1 Announce Type: cross Abstract: Disaggregating the two stages, Prefill and Decode, onto separate GPU pools is now a standard optimization for (text-only) LLM serving. However, multim

hardwarearxiv-cs-lg

28 Sept 2026

Enhancing Photogrammetric Digital Surface Models with Pretrained Diffusion Models and Multimodal Conditioning

arXiv:2609.31199v1 Announce Type: cross Abstract: Large-scale Digital Surface Models (DSMs) can be produced cost-effectively from satellite images via stereo-photogrammetry. However, the resulting 3D

28 Sept 2026

Skip the Talk, Re-Focus on Vision: Latent Reasoning for Reasoning Segmentation in Multimodal Large Language Models

arXiv:2609.30783v1 Announce Type: cross Abstract: Reasoning segmentation aims to interpret implicit textual queries and enable fine-grained visual perception, which is critical for applications such a

safetyarxiv-cs-ai

28 Sept 2026

SkinAgent AI: A Safety-Grounded Multimodal Agentic Framework for Non-Diagnostic Skincare Support

arXiv:2609.29341v1 Announce Type: new Abstract: Consumer-facing skincare AI must coordinate visual evidence, product information, tool use, and user-facing actions within explicit evidence and safety

25 Sept 2026

AstraLOD3: Zero-shot multimodal agentic reconstruction of LOD3 building models

arXiv:2609.28061v1 Announce Type: new Abstract: Automated LOD3 building modeling typically relies on purpose-built geometric or learning-based pipelines, limiting flexibility across heterogeneous buil

24 Sept 2026

Geometry-anchored PET-aware multimodal pseudo-CT synthesis for whole-body attenuation correction: the BIC-MAC Challenge

arXiv:2609.27848v1 Announce Type: new Abstract: The BIC-MAC challenge targets whole-body pseudo-CT synthesis from NAC-PET, Dixon MRI, and a 2D topogram for CT-less PET attenuation correction. We propo

24 Sept 2026

A Multimodal Large Language Model-Driven Framework for Context-Aware UAV Emergency Landing Site Selection

arXiv:2602.01163v2 Announce Type: replace Abstract: Safe UAV emergency landing requires more than just identifying flat terrain; it demands understanding complex semantic risks (e.g., crowds, temporar

Evidence-gated multimodal parsing and vectorization of architectural floor plans

arXiv:2609.25615v1 Announce Type: new Abstract: Architectural floor plans remain a high-friction barrier to archive digitization and early design-model preparation because heterogeneous graphics encod

From Token Importance to Conditional Removability: Rethinking Visual Token Pruning in Multimodal Large Language Models

arXiv:2609.26484v1 Announce Type: new Abstract: Training-free visual-token pruning often uses token importance, redundancy, or related selection criteria as proxies for safe removal. We show that thes

Applications

Moving6DPoSe: A Multimodal Database for Monocular 6D Pose Estimation and Segmentation of Moving Objects

arXiv:2609.26161v1 Announce Type: new Abstract: Estimating the 6D pose of moving objects remains challenging due to motion blur and the limited temporal resolution of conventional frame-based cameras.

applicationsarxiv-cs-cv

PartLLM: A Unified Multimodal Foundation for 3D Part Segmentation

arXiv:2609.25832v1 Announce Type: new Abstract: Part segmentation is a fundamental problem in computer graphics and 3D vision. Recent works have expanded 3D part segmentation beyond fixed taxonomies,

An Affordable AI-Integrated Smart Cane for Multimodal Mobility Assistance of Visually Impaired Users

arXiv:2609.22277v1 Announce Type: cross Abstract: Visual impairment affects over 2.2 billion people worldwide, yet conventional white canes cannot detect elevated hazards or provide semantic environme

Benchmarking Off-the-Shelf Multimodal AI Models Against Dermatologists on Patient-Captured Skin Images

arXiv:2609.24190v1 Announce Type: new Abstract: Artificial intelligence (AI) has advanced at a rapid pace in recent years. Initially, breakthroughs in large language models caught widespread attention

Detecting Phone-Induced Pedestrian Distraction via a Multimodal Fusion Transformer

arXiv:2609.23507v1 Announce Type: new Abstract: The increasing reliance on mobile phones has made phone-induced pedestrian distraction increasingly prevalent. Activities such as texting, watching vide

agentsarxiv-cs-cv

Document Retrieval-Aware Chunking (D-RAC): Universal Retrieval-Aware Ingestion of Enterprise Documents via PDF Normalization and Multimodal Markdown Conversion

arXiv:2609.24220v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats -- PDFs, Word documents, present

Graded-Relevance Composed Multimodal Retrieval for E-commerce Visual Search at Scale

arXiv:2609.24152v1 Announce Type: cross Abstract: Visual search on large e-commerce catalogs must serve both 'similarity' queries that ask for items resembling an uploaded image and 'modifier' queries

Applications

IDProxy: CTR Prediction with Multimodal LLMs for Cold-Start Recommendation at Xiaohongshu

arXiv:2603.01590v2 Announce Type: replace-cross Abstract: Content-driven platforms such as Xiaohongshu often leverage click-through rate (CTR) prediction models for recommendation. However, these mode

applicationsarxiv-cs-lg

Learning to Drive on Mars: Visual Multimodal Traversability Estimation for Off-World Navigation

arXiv:2609.24952v1 Announce Type: new Abstract: Autonomous navigation on Mars requires vehicles to distinguish between traversable terrains across diverse and visually challenging environments. Howeve

agentsarxiv-cs-ro

Local Ai

MarsRecon: Self-Supervised and Multimodal Surface Representations for Mars

arXiv:2609.22379v1 Announce Type: new Abstract: High-resolution orbital imagery offers a rich record of the Martian surface, but sparse geological labels limit supervised representation learning. We p

local-aiarxiv-cs-cv

Beyond Final Answers: CRYSTAL Benchmark for Transparent Multimodal Reasoning Evaluation

arXiv:2603.13099v3 Announce Type: replace Abstract: We introduce CRYSTAL (Clear Reasoning via Yielded Steps, Traceability, and Logic), a diagnostic benchmark with 6,372 instances that evaluates multim

21 Sept 2026

HERMES: A Holistic End-to-End Risk-Aware Multimodal Embodied System with Vision-Language Models for Long-Tail Autonomous Driving

arXiv:2602.00993v2 Announce Type: replace-cross Abstract: End-to-end autonomous driving models increasingly benefit from large vision-language models for semantic understanding, yet safe and reliable

21 Sept 2026

ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation

arXiv:2603.03279v2 Announce Type: replace-cross Abstract: Achieving autonomous and versatile whole-body loco-manipulation remains a central barrier to making humanoids practically useful. Yet existing

safetyarxiv-cs-cv

21 Sept 2026

A Unified Evaluation Framework for Trustworthy Large Language Models, Agentic AI, and Multimodal Systems

arXiv:2609.19524v1 Announce Type: new Abstract: Benchmark scores alone provide an incomplete basis for assessing the trustworthiness of modern artificial intelligence systems. Large language models (L

18 Sept 2026

Applications

Donate or Create? Comparing Data Collection Strategies for Emotion-labeled Multimodal Social Media Posts

arXiv:2505.24427v2 Announce Type: replace Abstract: Accurate modeling of subjective phenomena such as emotion expression requires data annotated with authors' intentions. Commonly such data is collect

applicationsarxiv-cs-cl

Future of general LLM work (interp/inference/alignment) vs agentic/physical AI (VLA, multimodal) for career [D]

I'm at a crossroads with two grad school options that would take me in somewhat different research directions, and I wanted some general advice on these fields, their growth, and industry alignment. I

safetyr-machinelearning

Hypothesis-Driven Autonomous Materials Synthesis with Multimodal LLM Agents

arXiv:2609.18598v1 Announce Type: cross Abstract: Self-driving laboratories can explore synthesis conditions autonomously, but their decision-making layer is typically a black-box optimizer, and the o

agentsarxiv-cs-ai

Multimodal Behavior Tree Generation: A Small Vision-Language Model for Robot Task Planning

model-releasesarxiv-cs-ro

arXiv:2603.06084v2 Announce Type: replace Abstract: Large language models have been widely used for robotic task planning, often taking advantage of representations such as Behavior Trees (BTs). Visio

Online Multimodal Workload Assessment in Contact-Rich Physical Human-Robot Interaction

arXiv:2609.18031v1 Announce Type: new Abstract: Contact-rich physical human--robot interaction (pHRI) imposes time-varying demands associated with physical interaction, motor regulation, and physiolog

researcharxiv-cs-ro

Robust Fault Detection in Mechanical Multimodal Time Series via Self-Supervised Cross-Modal Reconstruction

arXiv:2609.16314v1 Announce Type: new Abstract: Fault detection is essential in industrial systems, enabling early identification of abnormal behaviour and improving safety, reliability, and operation

safetyarxiv-cs-lg

16 Sept 2026

A Generative AI Integrated Multimodal Framework for Low-Latency Multi-Camera Person Re-Identification

arXiv:2609.14419v1 Announce Type: cross Abstract: Person re-identification (ReID) is essential for multi-camera surveillance and tracking, yet remains difficult due to viewpoint and illumination chang

researcharxiv-cs-ai

15 Sept 2026

Multimodal Foundation Models Adaptation based on Domain-Aware Relaxed Orthogonal Subspace for Remote Sensing

arXiv:2609.13654v1 Announce Type: new Abstract: Pretrained foundation models (FMs) have achieved remarkable success in computer vision, yet their high fine-tuning cost limits practical deployment. Par

15 Sept 2026

SlipSense: Multimodal Tactile Learning for Low-Latency and Generalized Slip Detection

arXiv:2609.15910v1 Announce Type: cross Abstract: Slip detection is fundamental to dexterous manipulation, yet existing systems often lack precise characterization of detection latency and cross-platf

researcharxiv-cs-ai

15 Sept 2026

CARLAverse: A Highly Modular, Distributed, and Multimodal Framework for Human-in-the-Loop Simulation

arXiv:2609.11478v1 Announce Type: new Abstract: The development of autonomous driving demands comprehensive testing in mixed-traffic scenarios involving vulnerable road users (VRUs), where purely arti

agentsarxiv-cs-ro

11 Sept 2026

ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs

model-releasesarxiv-cs-ro

arXiv:2609.10895v1 Announce Type: new Abstract: Reacting to sudden physical hazards (catching a slipping plate, dodging a falling knife) is both a meaningful test of embodied intelligence and a hard r

11 Sept 2026

Scale-Aware 3D Deep Learning for Robust Brain Metastasis Detection in Multimodal MRI

arXiv:2609.10825v1 Announce Type: cross Abstract: Detecting brain metastases in magnetic resonance imaging (MRI) remains challenging because lesions vary widely in size and appearance, with very small

11 Sept 2026

NormViz: A Benchmark and Framework for Grounding Multimodal Reasoning in Global Cultures

arXiv:2609.06831v1 Announce Type: new Abstract: AI systems are used worldwide, but they struggle to serve the needs of culturally diverse populations. Prior work on cultural understanding evaluates AI

10 Sept 2026

On-Policy Distillation for Vision-Language Model Adaptation, an Effective Paradigm on Low-Quality Multimodal Data

arXiv:2609.10321v1 Announce Type: new Abstract: Knowledge distillation offers an efficient route to transfer a task-adapted vision-language teacher to a compact student. The training target in current

safetyarxiv-cs-cl

10 Sept 2026

AAMBERS-UAV: Acquisition-Aware Multimodal Backbone Evaluation and Ranking for UAV Weedy Rice Segmentation

arXiv:2609.05762v1 Announce Type: new Abstract: UAV image collections contain spatially and temporally related frames, yet semantic-segmentation benchmarks commonly split them at image level. Such spl

9 Sept 2026

DYAD: A Multimodal Dataset of Co-Located Human Assistance

arXiv:2609.09023v1 Announce Type: new Abstract: An embodied assistant working beside a person must track task state, recognize help seeking, choose how to intervene, and produce an appropriate respons

safetyarxiv-cs-ro

9 Sept 2026

Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras

arXiv:2609.06928v1 Announce Type: new Abstract: Emotion analysis is a fundamental task in computer vision, but its practical deployment remains constrained by the privacy risks inherent to conventiona