Search: “multimodal”

Accurate and Scalable Multimodal Pathology Retrieval via Attentive Vision-Language Alignment

arXiv:2510.23224v2 Announce Type: replace Abstract: The rapid digitization of histopathology slides has opened new opportunities for computational tools in clinical and research workflows. Content-bas

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

EventVL: Understand Event Streams via Multimodal Large Language Model

DGX agent

arXiv:2501.13707v3 Announce Type: replace-cross Abstract: The event-based Vision-Language Model (VLM) recently has made good progress for practical vision tasks. However, most of these works just util

safetyarxiv-cs-ai

15 Sept 2026

Global-Local Contextual Progressive Expansion Network for Martian Landslide Segmentation in Multimodal Remote Sensing Imagery

arXiv:2609.13332v1 Announce Type: new Abstract: Automated landslide segmentation on Mars is one of the important tasks for understanding its surface processes, and all will aid in future space explora

Multimodal deep learning from spectra for small-molecule structure identification: enhancing robustness with mixed-condition training

arXiv:2609.14360v1 Announce Type: new Abstract: In practical molecular characterization, small-molecule structure identification benefits from complementary spectroscopic evidence, but missing, degrad

Through the Eyes of the Beholder: Biometric and Demographic Conditioning for Multimodal Sexism Detection

model-releasesarxiv-cs-ai

arXiv:2609.15608v1 Announce Type: cross Abstract: Detecting sexism on the internet is a fundamentally subjective task; our team, VANGUARD, addresses this challenge in the EXIST 2026 Task 2 by proposin

Evaluating Time-Series Foundation Models and Multimodal Dietary Context for CGM Forecasting

arXiv:2609.11872v1 Announce Type: cross Abstract: Continuous glucose monitoring (CGM) provides high-frequency measurements of glucose dynamics and enables short-term glucose forecasting for diabetes m

Multimodal Taxonomic Conditioning for Generative Plankton Imagery

arXiv:2609.11673v1 Announce Type: new Abstract: Automated plankton imaging produces severely long-tailed datasets, where the rare taxa of greatest ecological interest have too few images to train or e

ReGround: Grounding Reviewer Comments in Multimodal Evidence

arXiv:2609.11460v1 Announce Type: new Abstract: Reviewer comments naturally relate to specific parts of the reviewed paper, yet grounding these comments to the underlying evidence is difficult due to

researcharxiv-cs-cl

Temporal and Multimodal Deep Learning for Cyberattack Detection in LEO Satellite Systems

arXiv:2609.10746v1 Announce Type: cross Abstract: The growing reliance on Low-Earth Orbit (LEO) satellite communication systems has increased the need for intelligent methods capable of detecting cybe

Bridging the Semantic-Utility Gap in Multimodal RAG via Generator-in-the-Loop Alignment

arXiv:2609.08188v1 Announce Type: new Abstract: Vision-language models (VLMs) augmented with retrieval-augmented generation (RAG) benefit from access to external evidence. However, standard retrievers

safetyarxiv-cs-ai

10 Sept 2026

Local Ai

OmniMed-FL: A Robust Multimodal Federated Learning Framework for Clinical Diagnosis

arXiv:2609.10364v1 Announce Type: new Abstract: Simultaneous assessment of medical imaging and patient records is often required in clinical diagnosis. However, standard machine learning algorithms ca

local-aiarxiv-cs-lg

10 Sept 2026

Multimodal Resource-Exhaustion Attacks on Vision-Language Models via Joint Pixel-Prompt Optimization

arXiv:2609.05889v1 Announce Type: cross Abstract: Resource-exhaustion attacks against autoregressive vision-language models (VLMs) typically assume unimodal threat models, treating the image branch as

Segment Any Motion with Radar: Robust Multimodal Moving-Object Segmentation and Tracking

arXiv:2609.08346v1 Announce Type: new Abstract: Moving-object perception must decide which image regions correspond to real motion and keep every instance identified over time. Methods that read motio

Stable-MM-R1: Anchoring Multimodal Reasoning Dynamics via Entropy-Guided Stratification

arXiv:2609.07148v1 Announce Type: cross Abstract: While Reinforcement Learning (RL) effectively incentivizes reasoning in Large Language Models, current pipelines are hindered by training instability

The Price of Consistency: Exploiting Visual Anchors for Multimodal Jailbreaking in Video Generation

arXiv:2609.07216v1 Announce Type: new Abstract: The rapid evolution of video generation has shifted the paradigm from pure text-driven to multi-conditional controllable generation, with reference imag

VoT: Vision-of-Thought for Unified Multimodal Representation Alignment

arXiv:2609.07815v1 Announce Type: new Abstract: Current text-to-image systems typically employ a 'text encoder plus diffusion decoder' paradigm, in which text semantics directly modulate continuous la

Applications

Explainable Multimodal Deep Learning Integrating Imaging and Clinical Data for Oral Potentially Malignant Disorder Detection

arXiv:2609.04512v1 Announce Type: cross Abstract: Oral potentially malignant disorders (OPMDs) are critical precursors to oral cancer, yet clinical detection remains challenging because of substantial

applicationsarxiv-cs-cv

MEOX: Compact Multimodal Mixture-of-Experts for Earth Observation

arXiv:2609.05351v1 Announce Type: new Abstract: Recent advances in Earth Observation representation learning accommodate heterogeneous sensors and missing observations, often through larger architectu

MMTClinic: Multimodal, Multilingual Time Series Question Answering and Reasoning Benchmark for Clinical Domain

model-releasesarxiv-cs-ai

arXiv:2609.04842v1 Announce Type: cross Abstract: Time-series data in clinical settings is crucial for capturing dynamic changes in a patient's health over time, enabling timely diagnosis, personalize

SMILE: Self-Explainable Multimodal Information Bottleneck for Medical Diagnosis

arXiv:2609.05174v1 Announce Type: new Abstract: Explainability is increasingly seen as a crucial requirement in AI-based medical diagnosis, particularly in safety-critical clinical decision-making. Mo

Agents

YOLO with Kolmogorov-Arnold networks and vision-language foundation models for interpretable object detection with trustworthy multimodal AI in computer vision perception

arXiv:2603.23037v2 Announce Type: replace-cross Abstract: The trustworthy object detection capabilities of a novel Kolmogorov-Arnold network framework are examined here. The approach addresses a key l

agentsarxiv-cs-ai

MulDP: Multimodal Diffusion Policy for Autonomous Quadruped Parkour Navigation across Complex Terrains

arXiv:2609.03984v1 Announce Type: new Abstract: Quadruped robots have demonstrated impressive agility in parkour locomotion across complex terrains. However, most systems still rely on human intervent

safetyarxiv-cs-ro

4 Sept 2026

PIVOTSBench: Evaluating Fine-Grained Interpersonal Relationship Reasoning in Multimodal Large Language Models

model-releasesarxiv-cs-cl

arXiv:2606.23092v2 Announce Type: replace Abstract: Humans possess an innate ability to understand fine-grained interpersonal relationships, which is central to everyday social interactions. Although

3 Sept 2026

BrainDiff: Longitudinal Report Generation for Multimodal Brain MRI

arXiv:2609.00593v1 Announce Type: new Abstract: Neuroradiologists rarely read a brain MRI in isolation, yet automated brain-MRI report generation has been built almost entirely for single studies. Tem

2 Sept 2026

Multimodal RGB-Infrared Combination for UAV-Based Wildfire Segmentation: A Comparative Study on FLAME3

arXiv:2609.01390v1 Announce Type: new Abstract: Unmanned Aerial Vehicles (UAVs) have emerged as a promising platform for firefighting operations due to their flexibility, low operational cost, and abi

2 Sept 2026

Semantic-Guided Multimodal Preprocessing for Vision Transformer-Based Clear Cell Renal Cell Carcinoma Grading

arXiv:2609.01426v1 Announce Type: cross Abstract: Clear cell renal cell carcinoma (CCRCC) grading is essential for treatment planning, yet existing approaches either analyze patch-level images directl

2 Sept 2026

Lot Machine: Multimodal Lot Extraction from Auction Catalogs

model-releasesarxiv-cs-ai

arXiv:2608.30510v1 Announce Type: cross Abstract: For provenance research and art market studies, auction catalogs are an essential resource to trace specific objects over time and space. While histor

1 Sept 2026

Tutorials

Phantom-Insight: Adaptive Multi-cue Fusion for Video Camouflaged Object Detection with Multimodal LLM

arXiv:2509.06422v2 Announce Type: replace Abstract: Video camouflaged object detection (VCOD) is challenging due to dynamic environments. Existing methods face two main issues: (1) SAM-based methods s

tutorialsarxiv-cs-cv

1 Sept 2026

SurgSkill-Bench: A Benchmark for Multimodal Surgical Skill Assessment

arXiv:2608.30872v1 Announce Type: new Abstract: Objective assessment of surgical technical skill is important for surgical training and structured feedback, but current workflows remain dependent on l

1 Sept 2026

FENCE: A Financial and Multimodal Jailbreak Detection Dataset

arXiv:2602.18154v3 Announce Type: replace Abstract: Jailbreaking poses a significant risk to the deployment of Large Language Models (LLMs) and Vision Language Models (VLMs). VLMs are particularly vul

safetyarxiv-cs-cl

31 Aug 2026

Local Ai

Mixture of Multicenter Experts in Multimodal AI for Debiased Radiotherapy Target Delineation

arXiv:2410.00046v4 Announce Type: replace-cross Abstract: Clinical decision-making reflects diverse strategies shaped by regional patient populations and institutional protocols. However, most existin

local-aiarxiv-cs-cv

31 Aug 2026

Local Ai

A Lightweight Multimodal Vision-Language Framework for Early-Stage Anatomical Green Fruit Classification in Commercial Orchards

arXiv:2608.24935v1 Announce Type: new Abstract: Accurate identification of early-stage apple fruitlet anatomical structures, including the calyx, fruitlet body, and peduncle, is essential for robotic

local-aiarxiv-cs-cv

27 Aug 2026

MObyGaze: a film dataset of multimodal objectification densely annotated by experts

arXiv:2505.22084v2 Announce Type: replace Abstract: Characterizing and quantifying gender representation disparities in audiovisual storytelling contents is necessary to grasp how stereotypes may perp

27 Aug 2026

Action-Aligned Retrieval with Pairwise Multimodal Reranking for Text-Based Person Anomaly Search

arXiv:2608.23503v1 Announce Type: new Abstract: Text-based person anomaly search requires distinguishing individuals based on fine-grained, context-dependent behaviors rather than mere appearance. Exi

Modalities Should Talk to Each Other: Dual-Stream Multimodal Learning for Long-Horizon Influenza Forecasting

arXiv:2608.23373v1 Announce Type: new Abstract: Forecasting long-range influenza-like illness (ILI) matters for public health readiness. Publicly available surveillance datasets typically pair numeric

Real-TurnTurk: A Multimodal Turkish Corpus for Turn-Taking Prediction

arXiv:2608.22071v1 Announce Type: cross Abstract: Turn-taking is a basic organizational feature of human conversation and remains difficult to model in natural, synchronous dialog systems. While exist

Training-Free Pseudo-Fusion for Composed Image Retrieval with Diffusion Models and Multimodal Large Language Models

arXiv:2608.23102v1 Announce Type: new Abstract: Composed Image Retrieval (CIR) is an emerging paradigm in content-based image retrieval that enables users to formulate compositional queries by combini

UR^{2}-MLLM: Uncertainty-aware Revisit Reasoning in Multimodal Large Language Models for Radiology Report Generation

arXiv:2608.22217v1 Announce Type: new Abstract: Radiologists generate diagnostic reports through iterative and selective revisiting of suspicious regions to refine their interpretations. Recent multim

Masking Is Not Enough: Generative Restoration for Multimodal De-Identification in Medical AI

arXiv:2608.21133v1 Announce Type: new Abstract: Medical image-text data can expose protected health information (PHI) through both visible image content as well as accompanying text, creating a barrie

24 Aug 2026

Tutorials

DiffDef: A Diffusion Model for Generating Multimodal Goal Shapes From Demonstrations for Deformable Object Manipulation

arXiv:2506.18779v2 Announce Type: replace Abstract: Deformable object manipulation is a key capability in many robotic applications. A promising paradigm for this problem is shape servoing, which aims

tutorialsarxiv-cs-ro

21 Aug 2026

MUST-PET: MUltimodal Self-supervised learning across Tracers for whole-body PET/CT-based lesion segmentation

arXiv:2608.19666v1 Announce Type: new Abstract: Deep learning-based whole-body PET-CT lesion segmentation can support cancer staging, treatment planning, and response assessment, but generalization is

21 Aug 2026

DynCur-Geo: Dynamic Curiosity Reward Shaping for Multimodal Active Geo-Localization

arXiv:2608.18673v1 Announce Type: new Abstract: Active geo-localization enables low-altitude UAVs to search for specified targets from limited local aerial observations, supporting time-sensitive appl

20 Aug 2026

MS-MFAD : Multimodal large language models for Face Anti-spoofing Detection

arXiv:2608.17328v1 Announce Type: new Abstract: Facial biometric recognition systems currently face compound threats intertwining generative AI and high-fidelity physical spoofing. Existing defenses s

19 Aug 2026

Applications

LUNG-KGMM: Knowledge-Guided Multimodal Learning for Lung Cancer Incidence Prediction

arXiv:2608.14657v1 Announce Type: cross Abstract: Early identification of lung cancer risk is critical for timely intervention, yet existing prediction models are limited by their reliance on single d

applicationsarxiv-cs-cv

18 Aug 2026

Towards Efficient Multimodal and Multilingual Opinion Extraction for STI: A QLoRA-Based Fine-Tuning Approach

arXiv:2608.14152v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have reshaped semantic analysis. Opinion Extraction (OE) for Science and Technology Intelligence (STI) r

17 Aug 2026

Agents

Wyvern: An Agentic Framework for Generating Grounded Multimodal Reports

arXiv:2608.14446v1 Announce Type: new Abstract: In the current artificial intelligence-driven innovation era, the pace of knowledge growth is accelerating, and is hard to keep up with. While generativ

agentsarxiv-cs-ai

17 Aug 2026

The perfect way for Google to screw over OAI and Anthropic is by releasing a 120B dense multimodal Gemma model

model-releasesr-localllama

The two leading labs are already feeling extremely threatened by Qwen & friends, however I think there are tons of enterprises and organizations in the West that don't feel comfortable using Chinese m

15 Aug 2026

Represent, Then Generate: Multimodal-Conditioned Time-Series Generation under Irregular Missingness

arXiv:2608.12592v1 Announce Type: new Abstract: Continuous physiological time series underpin modern clinical monitoring, yet many of the most informative signals are invasive, expensive, or simply un