Search: “multimodal”

Scaling to Multimodal and Multichannel Heart Sound Classification with Synthetic and Augmented Biosignals

arXiv:2509.11606v4 Announce Type: replace-cross Abstract: Cardiovascular diseases (CVDs) are the leading cause of death worldwide, accounting for approximately 17.9 million deaths each year. Early det

researcharxiv-cs-lg

3 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Leveraging Multimodality for Real-Time Classification of Transients and Variables found by the Zwicky Transient Facility

DGX agent

arXiv:2607.00228v1 Announce Type: cross Abstract: Modern time-domain surveys such as the Zwicky Transient Facility (ZTF) generate hundreds of thousands of alerts each night, making real-time decisions

researcharxiv-cs-lg

2 Jul 2026

Towards Developing a Multimodal Chat Assistant for University Stakeholders: RAG-based Approach

arXiv:2607.01115v1 Announce Type: cross Abstract: University stakeholders often face difficulties in accessing timely and reliable information, especially in developing countries, where there are very

2 Jul 2026

CoMet: Context and Multiplicity Decomposition for Multimodal Uncertainty Estimation

arXiv:2606.32012v1 Announce Type: cross Abstract: Uncertainty estimation has been a long-standing challenge in AI models; it amounts to 'knowing what you don't know,' and metacognition is notoriously

1 Jul 2026

GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding

arXiv:2511.00810v4 Announce Type: replace-cross Abstract: Graphical user interface (GUI) grounding is a key capability for computer-use agents, mapping natural-language instructions to actionable regi

1 Jul 2026

IterCAD: An Iterative Multimodal Agent for Visually-Grounded CAD Generation and Editing

arXiv:2606.13368v2 Announce Type: replace Abstract: Computer-Aided Design is pivotal in modern manufacturing, yet existing automated methods predominantly rely on open-loop, one-shot generation, creat

1 Jul 2026

Multimodal Mathematical Reasoning with Diverse Solving Perspective

model-releasesarxiv-cs-cl

arXiv:2507.02804v2 Announce Type: replace Abstract: Recent progress in large-scale reinforcement learning (RL) has notably enhanced the reasoning capabilities of large language models (LLMs), especial

Multimodal Representation Alignment for Cross-modal Information Retrieval

arXiv:2506.08774v2 Announce Type: replace-cross Abstract: Different machine learning models can represent the same underlying concept in different ways. This variability is particularly valuable for i

See, Think, Learn: A Self-Taught Multimodal Reasoner

arXiv:2512.02456v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) have achieved remarkable progress in integrating visual perception with language understanding. However, effecti

tutorialsarxiv-cs-cl

StarDojo: Benchmarking Open-Ended Behaviors of Agentic Multimodal LLMs in Production-Living Simulations with Stardew Valley

arXiv:2507.07445v3 Announce Type: replace Abstract: Autonomous agents navigating human society must master both production activities and social interactions, yet existing benchmarks rarely evaluate t

Traffic-CBM: A Structurally Interpretable Multimodal Framework for Encrypted Traffic Classification

arXiv:2606.29909v1 Announce Type: new Abstract: Encrypted traffic classification has achieved strong performance, but its decision process remains difficult to interpret. Existing methods usually comb

tutorialsarxiv-cs-cv

Agents

Dialogue to Detection: A Multimodal Hybrid NLP Pipeline for Insurance Fraud Detection

arXiv:2606.28002v1 Announce Type: cross Abstract: Insurance fraud imposes substantial financial losses and operational inefficiencies, raising premiums and impacting trust among legitimate policyholde

agentsarxiv-cs-ai

29 Jun 2026

DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injection

arXiv:2606.27499v1 Announce Type: cross Abstract: Research on agent memory has matured rapidly, but almost entirely on the text side: few existing benchmarks ask, in an interactive environment, when a

29 Jun 2026

HiMu: Hierarchical Multimodal Frame Selection for Long Video Question Answering

arXiv:2603.18558v2 Announce Type: replace-cross Abstract: Long-form video question answering requires reasoning over extended temporal contexts, making frame selection a critical bottleneck for multi-

29 Jun 2026

Methane-Plume Segmentation From Hyperspectral Satellite Imagery Via Multimodal Deep Learning

arXiv:2606.26416v1 Announce Type: new Abstract: Efficient detection of methane plumes is crucial for understanding and mitigating global warming, as accurately identifying and segmenting them in earth

applicationsarxiv-cs-cv

26 Jun 2026

Risk-Aware Selective Multimodal Driver Monitoring with Driver-State World Modeling

arXiv:2606.26922v1 Announce Type: cross Abstract: Continuous driver monitoring in automated vehicles requires low-latency inference while avoiding unsafe decisions under uncertain driver states. Large

26 Jun 2026

Cross-Attention Multimodal Learning for Predicting Response to Neoadjuvant Imatinib in Gastrointestinal Stromal Tumors: A Multicenter Retrospective Study

arXiv:2606.25579v1 Announce Type: cross Abstract: Background: Response to neoadjuvant imatinib in gastrointestinal stromal tumors (GISTs) is highly variable and cannot be reliably predicted using curr

25 Jun 2026

Agents

MIL-LC: A Robust Magnetometer-Inertial-LiDAR Fusion Multimodal Localization Framework

arXiv:2606.25796v1 Announce Type: new Abstract: Localization in challenging environments, such as GNSS-denied, geometrically repetitive, or textureless scenes commonly found in offices, hotels, and un

agentsarxiv-cs-ro

25 Jun 2026

A Physics-Informed, Behavior-Aware Digital Twin for Robust Multimodal Forecasting of Core Body Temperature in Precision Livestock Farming

arXiv:2604.04098v2 Announce Type: replace Abstract: Precision livestock farming requires accurate and timely heat stress prediction to ensure animal welfare and optimize farm management. This study pr

applicationsarxiv-cs-cv

Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving

arXiv:2606.22913v1 Announce Type: new Abstract: Recent Vision-Language-Action (VLA) models have advanced end-to-end autonomous driving by incorporating reasoning for better interpretability and planni

Morphology-Aware Multimodal Representation Learning for Insect Phylogenetic Reconstruction

arXiv:2606.22077v1 Announce Type: new Abstract: Morphological traits provide important evidence for phylogenetic reconstruction and evolutionary relationship analysis. Recent image-based approaches ha

Odoriko: A Shape-Aware Multimodal Diffusion Framework for Human Motion

arXiv:2606.21135v1 Announce Type: new Abstract: Human motion generation has been widely studied across diverse input modalities, text, music, and video, and recent efforts have unified these into sing

Predicting Immune Biomarkers with MultiModal Mixture-of-Expert Pathology Foundation Models Empowers Precision Oncology

arXiv:2606.18123v2 Announce Type: replace Abstract: Predicting immune biomarkers associated with the tumor immune microenvironment (TIME) is critical for advancing precision oncology, yet existing app

DeceptionX: Explainable Deception Detection with Multimodal Large Language Models

arXiv:2606.11385v1 Announce Type: new Abstract: Deception detection is a critical and highly challenging task within affective computing and behavioral analysis. Existing deep learning methods typical

applicationsarxiv-cs-cv

11 Jun 2026

Decoding Multimodal Cues: Unveiling the Implicit Meaning Behind Hateful Videos

arXiv:2606.11953v1 Announce Type: new Abstract: Hateful videos have become prevalent on online platforms, highlighting an urgent need for effective detection. However, existing studies primarily focus

researcharxiv-cs-cl

11 Jun 2026

Agents

Data Journalist Agent: Transforming Data into Verifiable Multimodal Stories

arXiv:2606.11176v1 Announce Type: cross Abstract: Data tells stories that shape society; the data journalist's job is to turn raw information into stories non-experts can trust. A high-quality news fe

agentsarxiv-cs-cl

10 Jun 2026

AgroOmni: A Large-Scale Multi-view Agricultural Dataset for Cross-Scale Multimodal Reasoning

arXiv:2603.14342v2 Announce Type: replace-cross Abstract: Modern agricultural data is sourced from diverse platforms and spans multiple spatial scales, ranging from ground-level close-up photography t

Aligned but Not Partner-Specific: Distinguishing How Multimodal LLM Agents Succeed in Reference Games Without Human-Like Conventions

arXiv:2606.08081v1 Announce Type: cross Abstract: Repeated reference games test whether interlocutors replace their initially long descriptions with shorter, partner-specific conventions grounded in s

mllm-shap: A Shapley Value Explainability Platform for Text-Audio Multimodal Large Language Models

arXiv:2606.07531v1 Announce Type: cross Abstract: We introduce mllm-shap, an open-source Python framework designed to extend Shapley Value (SV) explainability from text-only Large Language Models to M

NutriMLLM: Multimodal Large Language Models for Dietary Micronutrient Analysis

arXiv:2606.08948v1 Announce Type: cross Abstract: Comprehensive estimation of dietary micronutrients from food images could improve clinical nutrition care, but training such models requires large mul

Local Ai

SMART: Shot-Aware Multimodal Video Moment Retrieval with Audio-Enhanced MLLM

arXiv:2511.14143v2 Announce Type: replace-cross Abstract: Video Moment Retrieval is a task in video understanding that aims to localize a specific temporal segment in an untrimmed video based on a nat

local-aiarxiv-cs-ai

Evidence-Based Intelligent Diagnostic and Therapeutic Visualization System with Large Language Models: Multi-Turn Interaction and Multimodal Treatment Plan Generation

arXiv:2606.06869v1 Announce Type: new Abstract: Aim: Existing AI-assisted traditional Chinese medicine diagnostic tools suffer from opaque reasoning processes, passive interaction, and limited treatme

applicationsarxiv-cs-ai

8 Jun 2026

From Vision to Text: A Compact Multimodal Approach for Robust, Cross-Domain Presentation Attack Detection on ID Cards

arXiv:2606.06966v1 Announce Type: new Abstract: Cross-domain shifts challenge Presentation Attack Detection (PAD) on ID Cards, given the restricted data available due to privacy concerns. This work pr

8 Jun 2026

Teaching the Way, Not the Answer: Privileged Tutoring Distillation for Multimodal Policy Optimization

arXiv:2606.07000v1 Announce Type: new Abstract: Recent post-training methods, particularly Reinforcement Learning with Verifiable Rewards (RLVR), have significantly enhanced the reasoning ability of L

8 Jun 2026

Biomazon: A Multimodal Dataset for 3D Forest Structure and Biomass Modeling in the Amazon Basin

arXiv:2606.05368v1 Announce Type: new Abstract: Accurate, spatially explicit characterization of tropical forest structure is essential for carbon accounting and ecosystem monitoring, yet most ML pipe

5 Jun 2026

Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding

model-releasesarxiv-cs-cl

arXiv:2505.05026v5 Announce Type: replace Abstract: User interface (UI) design goes beyond visuals to shape user experience (UX), underscoring the shift toward UI/UX as a unified concept. While recent

5 Jun 2026

ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation

arXiv:2606.05718v1 Announce Type: new Abstract: On-policy distillation (OPD) improves reasoning by training a student on trajectories sampled from its own policy under supervision from a teacher. In m

safetyarxiv-cs-cv

5 Jun 2026

ViewMask-1-to-3: Multi-View Consistent Image Generation via Multimodal Discrete Diffusion Models

arXiv:2512.14099v3 Announce Type: replace Abstract: Motivated by discrete diffusion's success in language-vision modeling, we explore its potential for multi-view generation, a task dominated by conti

4 Jun 2026

Enhancing Protein-Protein Interaction Prediction with Hierarchical Motif-based Multimodal Protein Embedding

arXiv:2606.02629v1 Announce Type: cross Abstract: Protein-protein interactions (PPIs) are essential for many biological processes. However, existing PPI prediction approaches suffer from two major lim

3 Jun 2026

Plan2Map: A Multimodal Benchmark for Document-Grounded Geospatial Boundary Reconstruction from Planning Records

arXiv:2606.02747v1 Announce Type: cross Abstract: Planning records define restrictions over geographic areas, but their source documents often provide only indirect spatial evidence rather than machin

3 Jun 2026

Interpretable Multimodal Gesture Recognition for Drone and Mobile Robot Teleoperation via Log-Likelihood Ratio Fusion

arXiv:2602.23694v3 Announce Type: replace-cross Abstract: Human operators are still frequently exposed to hazardous environments such as disaster zones and industrial facilities, where intuitive and r

Longitudinal Multimodal Sensing of Physical Activity and Well-Being in Older Adults

arXiv:2606.00345v1 Announce Type: new Abstract: Wearable and mobile sensing technologies enable continuous monitoring of human behavior and health in real-world settings. However, predictive modeling

safetyarxiv-cs-lg

ProductWebGen: Benchmarking Multimodal Product Webpage Generation

arXiv:2606.01022v1 Announce Type: cross Abstract: Crafting a product display webpage from a source product image, along with layout and visual content instructions, holds significant practical value f

Local Ai

Response-Aware Multimodal Learning for Post-Treatment Visual Acuity Forecasting

arXiv:2606.00588v1 Announce Type: new Abstract: Long-term visual acuity (VA) outcomes after anti-VEGF therapy are central to patient counseling, expectation setting, and follow-up planning in diabetic

local-aiarxiv-cs-cv

Towards Automated Discovery: A Review of Generative Models, Multimodal Learning and Closed-Loop Workflows in Inverse Materials Design

arXiv:2606.02507v1 Announce Type: cross Abstract: Inverse materials design is shifting materials discovery from forward prediction to targeted proposal of candidates that satisfy objectives under phys

tutorialsarxiv-cs-lg

Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVR

arXiv:2605.30912v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) improves vision-language models (VLMs) by optimizing outcome rewards derived from final answers.

tutorialsarxiv-cs-cl

1 Jun 2026

Local Ai

Best-Arm Identification-Based Trust Region Selection for Bayesian Optimization on Multimodal Functions

arXiv:2605.31050v1 Announce Type: new Abstract: Gaussian process-based Bayesian optimization (BO) is a popular approach for expensive black-box optimization, but its performance often degrades on comp

local-aiarxiv-cs-lg

1 Jun 2026

UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysis

arXiv:2510.15710v3 Announce Type: replace Abstract: Medical workflows routinely combine reading images with producing visual and textual outputs, making both image understanding and generation central