Search: “multimodal”

Evaluating Time-Series Foundation Models and Multimodal Dietary Context for CGM Forecasting

arXiv:2609.11872v1 Announce Type: cross Abstract: Continuous glucose monitoring (CGM) provides high-frequency measurements of glucose dynamics and enables short-term glucose forecasting for diabetes m

researcharxiv-cs-lg

11 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Multimodal Taxonomic Conditioning for Generative Plankton Imagery

DGX agent

arXiv:2609.11673v1 Announce Type: new Abstract: Automated plankton imaging produces severely long-tailed datasets, where the rare taxa of greatest ecological interest have too few images to train or e

model-releasesarxiv-cs-cv

11 Sept 2026

ReGround: Grounding Reviewer Comments in Multimodal Evidence

arXiv:2609.11460v1 Announce Type: new Abstract: Reviewer comments naturally relate to specific parts of the reviewed paper, yet grounding these comments to the underlying evidence is difficult due to

researcharxiv-cs-cl

11 Sept 2026

Temporal and Multimodal Deep Learning for Cyberattack Detection in LEO Satellite Systems

arXiv:2609.10746v1 Announce Type: cross Abstract: The growing reliance on Low-Earth Orbit (LEO) satellite communication systems has increased the need for intelligent methods capable of detecting cybe

researcharxiv-cs-lg

11 Sept 2026

Bridging the Semantic-Utility Gap in Multimodal RAG via Generator-in-the-Loop Alignment

arXiv:2609.08188v1 Announce Type: new Abstract: Vision-language models (VLMs) augmented with retrieval-augmented generation (RAG) benefit from access to external evidence. However, standard retrievers

safetyarxiv-cs-ai

10 Sept 2026

Local Ai

OmniMed-FL: A Robust Multimodal Federated Learning Framework for Clinical Diagnosis

arXiv:2609.10364v1 Announce Type: new Abstract: Simultaneous assessment of medical imaging and patient records is often required in clinical diagnosis. However, standard machine learning algorithms ca

local-aiarxiv-cs-lg

10 Sept 2026

Multimodal Resource-Exhaustion Attacks on Vision-Language Models via Joint Pixel-Prompt Optimization

arXiv:2609.05889v1 Announce Type: cross Abstract: Resource-exhaustion attacks against autoregressive vision-language models (VLMs) typically assume unimodal threat models, treating the image branch as

Segment Any Motion with Radar: Robust Multimodal Moving-Object Segmentation and Tracking

arXiv:2609.08346v1 Announce Type: new Abstract: Moving-object perception must decide which image regions correspond to real motion and keep every instance identified over time. Methods that read motio

Stable-MM-R1: Anchoring Multimodal Reasoning Dynamics via Entropy-Guided Stratification

arXiv:2609.07148v1 Announce Type: cross Abstract: While Reinforcement Learning (RL) effectively incentivizes reasoning in Large Language Models, current pipelines are hindered by training instability

The Price of Consistency: Exploiting Visual Anchors for Multimodal Jailbreaking in Video Generation

arXiv:2609.07216v1 Announce Type: new Abstract: The rapid evolution of video generation has shifted the paradigm from pure text-driven to multi-conditional controllable generation, with reference imag

VoT: Vision-of-Thought for Unified Multimodal Representation Alignment

arXiv:2609.07815v1 Announce Type: new Abstract: Current text-to-image systems typically employ a 'text encoder plus diffusion decoder' paradigm, in which text semantics directly modulate continuous la

Applications

Explainable Multimodal Deep Learning Integrating Imaging and Clinical Data for Oral Potentially Malignant Disorder Detection

arXiv:2609.04512v1 Announce Type: cross Abstract: Oral potentially malignant disorders (OPMDs) are critical precursors to oral cancer, yet clinical detection remains challenging because of substantial

applicationsarxiv-cs-cv

MEOX: Compact Multimodal Mixture-of-Experts for Earth Observation

arXiv:2609.05351v1 Announce Type: new Abstract: Recent advances in Earth Observation representation learning accommodate heterogeneous sensors and missing observations, often through larger architectu

MMTClinic: Multimodal, Multilingual Time Series Question Answering and Reasoning Benchmark for Clinical Domain

arXiv:2609.04842v1 Announce Type: cross Abstract: Time-series data in clinical settings is crucial for capturing dynamic changes in a patient's health over time, enabling timely diagnosis, personalize

SMILE: Self-Explainable Multimodal Information Bottleneck for Medical Diagnosis

arXiv:2609.05174v1 Announce Type: new Abstract: Explainability is increasingly seen as a crucial requirement in AI-based medical diagnosis, particularly in safety-critical clinical decision-making. Mo

Agents

YOLO with Kolmogorov-Arnold networks and vision-language foundation models for interpretable object detection with trustworthy multimodal AI in computer vision perception

arXiv:2603.23037v2 Announce Type: replace-cross Abstract: The trustworthy object detection capabilities of a novel Kolmogorov-Arnold network framework are examined here. The approach addresses a key l

agentsarxiv-cs-ai

MulDP: Multimodal Diffusion Policy for Autonomous Quadruped Parkour Navigation across Complex Terrains

arXiv:2609.03984v1 Announce Type: new Abstract: Quadruped robots have demonstrated impressive agility in parkour locomotion across complex terrains. However, most systems still rely on human intervent

safetyarxiv-cs-ro

4 Sept 2026

PIVOTSBench: Evaluating Fine-Grained Interpersonal Relationship Reasoning in Multimodal Large Language Models

model-releasesarxiv-cs-cl

arXiv:2606.23092v2 Announce Type: replace Abstract: Humans possess an innate ability to understand fine-grained interpersonal relationships, which is central to everyday social interactions. Although

3 Sept 2026

BrainDiff: Longitudinal Report Generation for Multimodal Brain MRI

arXiv:2609.00593v1 Announce Type: new Abstract: Neuroradiologists rarely read a brain MRI in isolation, yet automated brain-MRI report generation has been built almost entirely for single studies. Tem

2 Sept 2026

Multimodal RGB-Infrared Combination for UAV-Based Wildfire Segmentation: A Comparative Study on FLAME3

arXiv:2609.01390v1 Announce Type: new Abstract: Unmanned Aerial Vehicles (UAVs) have emerged as a promising platform for firefighting operations due to their flexibility, low operational cost, and abi

2 Sept 2026

Semantic-Guided Multimodal Preprocessing for Vision Transformer-Based Clear Cell Renal Cell Carcinoma Grading

arXiv:2609.01426v1 Announce Type: cross Abstract: Clear cell renal cell carcinoma (CCRCC) grading is essential for treatment planning, yet existing approaches either analyze patch-level images directl

2 Sept 2026

Lot Machine: Multimodal Lot Extraction from Auction Catalogs

arXiv:2608.30510v1 Announce Type: cross Abstract: For provenance research and art market studies, auction catalogs are an essential resource to trace specific objects over time and space. While histor

1 Sept 2026

Tutorials

Phantom-Insight: Adaptive Multi-cue Fusion for Video Camouflaged Object Detection with Multimodal LLM

arXiv:2509.06422v2 Announce Type: replace Abstract: Video camouflaged object detection (VCOD) is challenging due to dynamic environments. Existing methods face two main issues: (1) SAM-based methods s

tutorialsarxiv-cs-cv

1 Sept 2026

SurgSkill-Bench: A Benchmark for Multimodal Surgical Skill Assessment

arXiv:2608.30872v1 Announce Type: new Abstract: Objective assessment of surgical technical skill is important for surgical training and structured feedback, but current workflows remain dependent on l

1 Sept 2026

FENCE: A Financial and Multimodal Jailbreak Detection Dataset

arXiv:2602.18154v3 Announce Type: replace Abstract: Jailbreaking poses a significant risk to the deployment of Large Language Models (LLMs) and Vision Language Models (VLMs). VLMs are particularly vul

safetyarxiv-cs-cl

31 Aug 2026

Local Ai

Mixture of Multicenter Experts in Multimodal AI for Debiased Radiotherapy Target Delineation

arXiv:2410.00046v4 Announce Type: replace-cross Abstract: Clinical decision-making reflects diverse strategies shaped by regional patient populations and institutional protocols. However, most existin

local-aiarxiv-cs-cv

31 Aug 2026

Local Ai

A Lightweight Multimodal Vision-Language Framework for Early-Stage Anatomical Green Fruit Classification in Commercial Orchards

arXiv:2608.24935v1 Announce Type: new Abstract: Accurate identification of early-stage apple fruitlet anatomical structures, including the calyx, fruitlet body, and peduncle, is essential for robotic

local-aiarxiv-cs-cv

27 Aug 2026

MObyGaze: a film dataset of multimodal objectification densely annotated by experts

arXiv:2505.22084v2 Announce Type: replace Abstract: Characterizing and quantifying gender representation disparities in audiovisual storytelling contents is necessary to grasp how stereotypes may perp

27 Aug 2026

Action-Aligned Retrieval with Pairwise Multimodal Reranking for Text-Based Person Anomaly Search

arXiv:2608.23503v1 Announce Type: new Abstract: Text-based person anomaly search requires distinguishing individuals based on fine-grained, context-dependent behaviors rather than mere appearance. Exi

Modalities Should Talk to Each Other: Dual-Stream Multimodal Learning for Long-Horizon Influenza Forecasting

arXiv:2608.23373v1 Announce Type: new Abstract: Forecasting long-range influenza-like illness (ILI) matters for public health readiness. Publicly available surveillance datasets typically pair numeric

Real-TurnTurk: A Multimodal Turkish Corpus for Turn-Taking Prediction

arXiv:2608.22071v1 Announce Type: cross Abstract: Turn-taking is a basic organizational feature of human conversation and remains difficult to model in natural, synchronous dialog systems. While exist

Training-Free Pseudo-Fusion for Composed Image Retrieval with Diffusion Models and Multimodal Large Language Models

arXiv:2608.23102v1 Announce Type: new Abstract: Composed Image Retrieval (CIR) is an emerging paradigm in content-based image retrieval that enables users to formulate compositional queries by combini

UR^{2}-MLLM: Uncertainty-aware Revisit Reasoning in Multimodal Large Language Models for Radiology Report Generation

arXiv:2608.22217v1 Announce Type: new Abstract: Radiologists generate diagnostic reports through iterative and selective revisiting of suspicious regions to refine their interpretations. Recent multim

Masking Is Not Enough: Generative Restoration for Multimodal De-Identification in Medical AI

arXiv:2608.21133v1 Announce Type: new Abstract: Medical image-text data can expose protected health information (PHI) through both visible image content as well as accompanying text, creating a barrie

24 Aug 2026

Tutorials

DiffDef: A Diffusion Model for Generating Multimodal Goal Shapes From Demonstrations for Deformable Object Manipulation

arXiv:2506.18779v2 Announce Type: replace Abstract: Deformable object manipulation is a key capability in many robotic applications. A promising paradigm for this problem is shape servoing, which aims

tutorialsarxiv-cs-ro

21 Aug 2026

MUST-PET: MUltimodal Self-supervised learning across Tracers for whole-body PET/CT-based lesion segmentation

arXiv:2608.19666v1 Announce Type: new Abstract: Deep learning-based whole-body PET-CT lesion segmentation can support cancer staging, treatment planning, and response assessment, but generalization is

21 Aug 2026

DynCur-Geo: Dynamic Curiosity Reward Shaping for Multimodal Active Geo-Localization

arXiv:2608.18673v1 Announce Type: new Abstract: Active geo-localization enables low-altitude UAVs to search for specified targets from limited local aerial observations, supporting time-sensitive appl

20 Aug 2026

MS-MFAD : Multimodal large language models for Face Anti-spoofing Detection

arXiv:2608.17328v1 Announce Type: new Abstract: Facial biometric recognition systems currently face compound threats intertwining generative AI and high-fidelity physical spoofing. Existing defenses s

19 Aug 2026

Applications

LUNG-KGMM: Knowledge-Guided Multimodal Learning for Lung Cancer Incidence Prediction

arXiv:2608.14657v1 Announce Type: cross Abstract: Early identification of lung cancer risk is critical for timely intervention, yet existing prediction models are limited by their reliance on single d

applicationsarxiv-cs-cv

18 Aug 2026

Towards Efficient Multimodal and Multilingual Opinion Extraction for STI: A QLoRA-Based Fine-Tuning Approach

arXiv:2608.14152v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have reshaped semantic analysis. Opinion Extraction (OE) for Science and Technology Intelligence (STI) r

17 Aug 2026

Agents

Wyvern: An Agentic Framework for Generating Grounded Multimodal Reports

arXiv:2608.14446v1 Announce Type: new Abstract: In the current artificial intelligence-driven innovation era, the pace of knowledge growth is accelerating, and is hard to keep up with. While generativ

agentsarxiv-cs-ai

17 Aug 2026

Represent, Then Generate: Multimodal-Conditioned Time-Series Generation under Irregular Missingness

arXiv:2608.12592v1 Announce Type: new Abstract: Continuous physiological time series underpin modern clinical monitoring, yet many of the most informative signals are invasive, expensive, or simply un

researcharxiv-cs-lg

14 Aug 2026

Do You See What You Draw? A Semantic Closed-Loop Framework for Holistic Evaluation of Unified Multimodal Models

arXiv:2608.11907v1 Announce Type: cross Abstract: As Large Vision-Language Models increasingly aim to integrate visual generation and understanding within a single parameter space, evaluating such str

Applications

FEWT: Frequency-Enhanced Wavelet-based Transformer for Multimodal Wheeled Bimanual Manipulation

arXiv:2509.11109v4 Announce Type: replace Abstract: Embodied intelligence bridges the physical world and information spaces, with robots demonstrating immense potential through imitation learning algo

applicationsarxiv-cs-ro

Learning from Multimodal Pseudo-Labels for Robust Open-Vocabulary Instance and Panoptic Segmentation

arXiv:2608.11681v1 Announce Type: cross Abstract: This work addresses the challenge of open-vocabulary instance segmentation (OVIS) and open-set panoptic segmentation (OSPS), which aim to recognize bo

safetyarxiv-cs-ai

P2MFDS: A Privacy-Preserving Multimodal Fall Detection System for Elderly People in Bathroom Environments

arXiv:2506.17332v2 Announce Type: replace-cross Abstract: By 2050, people aged 65 and over are projected to make up 16% of the global population. As aging is closely associated with increased fall ris

MMArt A Multi-Perspective Multimodal Dataset for Visual Art Understanding

arXiv:2608.10706v1 Announce Type: new Abstract: Recent vision-language models demonstrate impressive general visual understanding, yet their art interpretation remains shallow: they describe surface c

12 Aug 2026

VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?