Search: “multimodal”

MAIA: Anticipating Interaction Anomalies in Contact-Rich Manipulation via Multimodal World Models

arXiv:2610.21686v1 Announce Type: new Abstract: Reliable contact-rich manipulation requires detecting execution deviations early enough for corrective action. Yet most monitoring methods are reactive,

safetyarxiv-cs-ro

16 Oct 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

ClinEdge: An Expert-Graded Benchmark of Real Patient Edge Cases Where Frontier Multimodal Models Fail

DGX agent

arXiv:2610.15239v1 Announce Type: new Abstract: Most benchmarks for large language models in medicine are built from licensing examinations and theoretical questions. The few that draw on real clinica

model-releasesarxiv-cs-ai

15 Oct 2026

OT-FlowRec: Optimal Transport-Guided Multi-Bridge Flow for Multimodal Sequential Recommendation

arXiv:2610.16038v1 Announce Type: cross Abstract: E-commerce user preferences evolve dynamically within sessions, exhibiting concurrent and multi-centric interest distributions. While existing multimo

15 Oct 2026

From a Large-Scale Teacher to Efficient Multimodal Students for Search Relevance Prediction

arXiv:2610.16569v1 Announce Type: cross Abstract: Relevance prediction between user queries and items is a fundamental component of e-commerce search systems. Traditionally, high-quality relevance lab

applicationsarxiv-cs-lg

MERIT: Multimodal Evidence Reasoning over Image Tables

arXiv:2610.17937v1 Announce Type: new Abstract: Table reasoning with language models typically assumes access to clean textual tables. In practice, tables are often serialized into text, which can cor

Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied Intelligence

arXiv:2610.18447v1 Announce Type: cross Abstract: World models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the f

VisExMEM: Visual Evidence-Based Explainable Multimodal Evaluation Metric

arXiv:2610.16441v1 Announce Type: new Abstract: Current image-text evaluation often relies on a global score, which gives limited insight into which semantic details are supported by the image. Explai

Locate, Judge, Read: Role-Specialized Tool Agents over a Knowledge-Point Index for Open-Domain Multimodal Document QA

arXiv:2610.13128v1 Announce Type: new Abstract: Question answering over large heterogeneous document collections requires first finding the right passages and then reading them, yet current retrieval-

12 Oct 2026

Reading the Whole Heart: Latent-Attention Masked Autoencoders for Multimodal Cardiac Representation Learning

arXiv:2609.12035v3 Announce Type: replace Abstract: Cardiovascular diagnosis and treatment rest on integrating complementary modalities, such as electrocardiogram, echocardiography, and chest X-rays,

9 Oct 2026

A method for multimodal analysis of TAIGA experiment data using essential features

arXiv:2610.08985v1 Announce Type: cross Abstract: The aim of processing and analyzing experimental data from physical experiments is to obtain physically significant information about the phenomenon u

8 Oct 2026

MemPilot: Orchestrating On-Demand Multimodal Memory Curation for LLM Agents

arXiv:2610.06830v1 Announce Type: cross Abstract: Memory has become integral to the LLM agent ecosystem, supporting information retention and reuse across interactions. However, most existing agent me

safetyarxiv-cs-lg

6 Oct 2026

Structural-Functional Brain Connectivity Generation via Multimodal Hypergraph-based Flow Matching

arXiv:2610.02722v1 Announce Type: new Abstract: Structural connectivity (SC) and functional connectivity (FC) provide complementary information on interactions between brain regions and are widely use

5 Oct 2026

Beyond Text: LLM-Based Dimensional Emotion Evaluation in Multimodal Dialogue

arXiv:2609.39072v1 Announce Type: cross Abstract: Emotion recognition in conversation has been widely studied, but applying Large Language Models (LLMs) to continuous dimensional emotion evaluation in

1 Oct 2026

Data-Driven Priors for Uncertainty-Aware Risk Prediction of Clinical Deterioration using Multimodal Data

arXiv:2603.08459v2 Announce Type: replace Abstract: Safe predictions are a crucial requirement for integrating predictive models into clinical decision support systems. One approach to improving trust

applicationsarxiv-cs-lg

1 Oct 2026

MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting

arXiv:2609.38523v1 Announce Type: cross Abstract: Financial forecasting from earnings conference calls requires models to reason over complex corporate disclosures, market expectations, and subtle com

1 Oct 2026

Agents

EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory

arXiv:2609.37923v1 Announce Type: new Abstract: Agents can learn from past executions, but enabling different agents to reuse and build on one another's experience remains challenging. We introduce Ep

agentsarxiv-cs-cv

Rethinking Multimodal Fake News Detection in the Generative AI Era

arXiv:2609.36850v1 Announce Type: new Abstract: Generative content is increasingly entering the production and dissemination of news, transforming fake news from manually fabricated or simply manipula

applicationsarxiv-cs-cl

Towards Breaking the Learning System Wall Using Multimodal Tutoring Transcriptions

arXiv:2609.36502v1 Announce Type: cross Abstract: Past research using log data has faced the 'learning system wall,' whereby few methods exist for generalizing models of student learning across platfo

VesselBench-800K: A Large-scale Perception Benchmark for Multimodal Vessel Detection, Counting, and Density Estimation

arXiv:2609.37003v1 Announce Type: new Abstract: Vessel perception from space is crucial for a wide range of maritime applications, from traffic monitoring to environmental protection. However, most ex

Visual sensitivity is not claim retractability: persistence-aware credit assignment for multimodal reinforcement learning

arXiv:2609.36572v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has been extended to Large Vision-Language Models (LVLMs), and perception-aware methods further en

Admissible Diffusion for Multimodal Interventional Trajectories

arXiv:2609.34433v1 Announce Type: new Abstract: Generating a plausible clinical trajectory does not establish what would happen under a different treatment. We present ADMIT, a framework combining irr

Advancing Wildlife Conservation through Multimodal Animal Re-Identification with Environmental Metadata

arXiv:2609.34094v1 Announce Type: new Abstract: Identifying individual animals is crucial for effective wildlife monitoring and conservation efforts. Recent advancements in computer vision have shown

Multimodal LLMs Outperform Pathology Foundation Models in Cross-Domain Histological Similarity

arXiv:2609.32876v1 Announce Type: new Abstract: State-of-the-art pathology foundation models, trained on millions of histology tiles, can fail to preserve tissue similarity when comparisons cross slid

SciGen-Verifier: A Multimodal Reasoner for Explainable Verification in Scientific Image Generation

arXiv:2609.33399v1 Announce Type: new Abstract: In realistic education, a solution is often expressed not only in words but in a drawing--a circuit, a geometric construction, a function plot--and a te

Agents

SegBanana: Steering Unified Multimodal Models into Medical Segmenters

arXiv:2609.34235v1 Announce Type: new Abstract: Medical image segmentation remains challenging in practical deployment, as models often struggle to generalize beyond the distributions covered by their

agentsarxiv-cs-cv

A Multimodal Dataset for Survival Prediction in Resected Pancreatic Ductal Adenocarcinoma

arXiv:2609.29726v1 Announce Type: new Abstract: Survival research in pancreatic ductal adenocarcinoma (PDAC) is limited by the scarcity of datasets linking whole-slide histology with clinical, molecul

25 Sept 2026

C3M: Cross-Session Multimodal Memory Maintenance for Long-Horizon Tasks

arXiv:2609.29735v1 Announce Type: new Abstract: Long-horizon tasks require preserving and later recovering cross-session evidence under a bounded, query-blind memory budget. Existing compression can d

25 Sept 2026

Implicit Behavior Coordination from Sub-Task Demonstrations by Exploiting Overlap-Induced Multimodality

arXiv:2607.09234v2 Announce Type: replace Abstract: Long-horizon robotic rearrangement is commonly formulated as a skill-sequencing problem, where distinct behaviors are explicitly represented and coo

safetyarxiv-cs-ro

25 Sept 2026

A Leakage-Aware Multimodal Evaluation Framework for Early Intraoperative Acute Kidney Injury Prediction

arXiv:2609.26848v1 Announce Type: cross Abstract: Postoperative acute kidney injury (AKI) after major non-cardiac surgery carries substantial morbidity, yet early intraoperative risk stratification re

24 Sept 2026

PRISM-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Tobacco Product and Legislative Policy Reasoning

arXiv:2609.23769v1 Announce Type: new Abstract: The disambiguation of semantically similar statutory text across jurisdictions is a retrieval problem that existing methods do not solve. This inter-con

22 Sept 2026

Local Ai

RoboTalk: Learning Multi-Robot Communication and Coordination from Multimodal Demonstrations

arXiv:2609.23997v1 Announce Type: new Abstract: Multi-robot collaboration could enable more efficient and scalable solutions to complex robotic tasks, but collaboration under partial observability rem

local-aiarxiv-cs-ro

22 Sept 2026

TAC-Time: Texts as Channels For Multimodal Time Series Forecasting

arXiv:2609.24156v1 Announce Type: new Abstract: Most existing time series forecasting methods rely solely on numerical observations, overlooking rich contextual information from auxiliary texts. Recen

safetyarxiv-cs-cl

22 Sept 2026

E-AVI: Evidence-Grounded Multimodal Assessment for Automated Video Interviews

arXiv:2609.20001v1 Announce Type: new Abstract: Automated video interview assessment integrates verbal content, acoustic delivery, and visual behavior, yet numerical predictions alone provide limited

Integrating knowledge from case reports: a medical ontology based multimodal information system with structured summary

arXiv:2609.19775v1 Announce Type: new Abstract: Published medical case reports serve as a crucial medical information carrier, documenting discoveries in rare diseases, diagnostic methods, and innovat

Lens: Bringing the Right Semantic Perspective into Focus for Training-Free Multimodal Representation Learning

arXiv:2609.20252v1 Announce Type: cross Abstract: High-quality representations are essential for a wide range of downstream tasks. Dedicated embedding models are explicitly optimized for representatio

OHRID-Retail: An Open Multimodal Dataset of Human Activity in Retail Environments

arXiv:2609.19302v1 Announce Type: new Abstract: Open datasets describing human behavior in environments shared with mobile robots remain limited, particularly for retail activities that combine locomo

researcharxiv-cs-ro

To Memories and Beyond: From Remembering to Knowing You across Long-Term Multimodal Personal Archives

model-releasesarxiv-cs-cl

arXiv:2609.19167v1 Announce Type: new Abstract: As AI systems evolve into personalized digital companions, a central capability is reasoning over a user's long-term personal history: not merely storin

Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models

model-releasesarxiv-cs-cl

arXiv:2412.16555v4 Announce Type: replace Abstract: Large language models (LLMs) are widely applied in various fields of society due to their powerful reasoning, understanding, and generation capabili

17 Sept 2026

Multi-Session Multimodal Underwater Mapping with Acoustic and Optical Imaging

arXiv:2609.17929v1 Announce Type: new Abstract: Accurate seafloor mapping is essential for marine science, archaeology, and environmental monitoring. However, integrating data from different sensors,

safetyarxiv-cs-ro

17 Sept 2026

A multimodal large language model for evidence-based autism spectrum disorder screening

arXiv:2609.16464v1 Announce Type: new Abstract: The clinical management of autism spectrum disorder (ASD) faces a bottleneck in early screening, mainly because trained specialists are scarce and conve

A Vision-Language Foundation Model for Precise and Comprehensive Brain Tumor Diagnosis from Preoperative Multimodal Data

arXiv:2609.16597v1 Announce Type: cross Abstract: Background Non-invasive presurgical diagnosis of brain tumor types from Magnetic Resonance Imaging (MRI) is essential but challenging due to overlappi

applicationsarxiv-cs-ai

MUUNRiver-Bench: Diagnosing Relation-Dependent Music Retrieval with Multimodal Instructions

arXiv:2609.16090v1 Announce Type: cross Abstract: Music retrieval is relation-dependent: given a reference track, a listener may seek its style with a new theme, a cover, or a comparable voice, and th

Local Ai

RepoAtlas: Guiding Coding Agents via Evolving Multimodal Repository Views

arXiv:2609.16936v1 Announce Type: cross Abstract: Large language model (LLM)-powered coding agents have made rapid progress in automating software engineering tasks, yet repository-level issue resolut

local-aiarxiv-cs-ai

Accurate and Scalable Multimodal Pathology Retrieval via Attentive Vision-Language Alignment

arXiv:2510.23224v2 Announce Type: replace Abstract: The rapid digitization of histopathology slides has opened new opportunities for computational tools in clinical and research workflows. Content-bas

EventVL: Understand Event Streams via Multimodal Large Language Model

arXiv:2501.13707v3 Announce Type: replace-cross Abstract: The event-based Vision-Language Model (VLM) recently has made good progress for practical vision tasks. However, most of these works just util

safetyarxiv-cs-ai

Global-Local Contextual Progressive Expansion Network for Martian Landslide Segmentation in Multimodal Remote Sensing Imagery

arXiv:2609.13332v1 Announce Type: new Abstract: Automated landslide segmentation on Mars is one of the important tasks for understanding its surface processes, and all will aid in future space explora

Multimodal deep learning from spectra for small-molecule structure identification: enhancing robustness with mixed-condition training

arXiv:2609.14360v1 Announce Type: new Abstract: In practical molecular characterization, small-molecule structure identification benefits from complementary spectroscopic evidence, but missing, degrad

Through the Eyes of the Beholder: Biometric and Demographic Conditioning for Multimodal Sexism Detection

arXiv:2609.15608v1 Announce Type: cross Abstract: Detecting sexism on the internet is a fundamentally subjective task; our team, VANGUARD, addresses this challenge in the EXIST 2026 Task 2 by proposin