Search: “multimodal”

Frailty Estimation in Elderly Oncology Patients Using Multimodal Wearable Data and Multi-Instance Learning

arXiv:2604.06985v1 Announce Type: cross Abstract: Frailty and functional decline strongly influence treatment tolerance and outcomes in older patients with cancer, yet assessment is typically limited

applicationsarxiv-cs-ai

10 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Synthetic Homes: A Multimodal Generative AI Pipeline for Residential Building Data Generation under Data Scarcity

DGX agent

arXiv:2509.09794v4 Announce Type: replace Abstract: Computational models have emerged as powerful tools for multi-scale energy modeling research at the building and urban scale, supporting data-driven

model-releasesarxiv-cs-ai

10 Apr 2026

XR-CareerAssist: An Immersive Platform for Personalised Career Guidance Leveraging Extended Reality and Multimodal AI

arXiv:2604.06901v1 Announce Type: cross Abstract: Conventional career guidance platforms rely on static, text-driven interfaces that struggle to engage users or deliver personalised, evidence-based in

10 Apr 2026

Agents

Highlights: 👉 Configurable thinking mode for step-by-step reasoning 👉 Multimodal understanding with text and image input, including docume…

Highlights: 👉 Configurable thinking mode for step-by-step reasoning 👉 Multimodal understanding with text and image input, including document parsing and OCR 👉 Native function calling with structured t

agentstogether-ai--x

9 Apr 2026

AI Knows Where You Are: Exposure, Bias, and Inference in Multimodal Geolocation with KoreaGEO

arXiv:2506.03371v2 Announce Type: replace Abstract: Recent advances in vision-language models (VLMs) have enabled accurate image-based geolocation, raising serious concerns about location privacy risk

16 Oct 2026

MAIA: Anticipating Interaction Anomalies in Contact-Rich Manipulation via Multimodal World Models

arXiv:2610.21686v1 Announce Type: new Abstract: Reliable contact-rich manipulation requires detecting execution deviations early enough for corrective action. Yet most monitoring methods are reactive,

safetyarxiv-cs-ro

16 Oct 2026

ClinEdge: An Expert-Graded Benchmark of Real Patient Edge Cases Where Frontier Multimodal Models Fail

arXiv:2610.15239v1 Announce Type: new Abstract: Most benchmarks for large language models in medicine are built from licensing examinations and theoretical questions. The few that draw on real clinica

15 Oct 2026

OT-FlowRec: Optimal Transport-Guided Multi-Bridge Flow for Multimodal Sequential Recommendation

arXiv:2610.16038v1 Announce Type: cross Abstract: E-commerce user preferences evolve dynamically within sessions, exhibiting concurrent and multi-centric interest distributions. While existing multimo

15 Oct 2026

From a Large-Scale Teacher to Efficient Multimodal Students for Search Relevance Prediction

arXiv:2610.16569v1 Announce Type: cross Abstract: Relevance prediction between user queries and items is a fundamental component of e-commerce search systems. Traditionally, high-quality relevance lab

applicationsarxiv-cs-lg

MERIT: Multimodal Evidence Reasoning over Image Tables

arXiv:2610.17937v1 Announce Type: new Abstract: Table reasoning with language models typically assumes access to clean textual tables. In practice, tables are often serialized into text, which can cor

safetyarxiv-cs-cv

Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied Intelligence

arXiv:2610.18447v1 Announce Type: cross Abstract: World models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the f

safetyarxiv-cs-cv

VisExMEM: Visual Evidence-Based Explainable Multimodal Evaluation Metric

arXiv:2610.16441v1 Announce Type: new Abstract: Current image-text evaluation often relies on a global score, which gives limited insight into which semantic details are supported by the image. Explai

Locate, Judge, Read: Role-Specialized Tool Agents over a Knowledge-Point Index for Open-Domain Multimodal Document QA

arXiv:2610.13128v1 Announce Type: new Abstract: Question answering over large heterogeneous document collections requires first finding the right passages and then reading them, yet current retrieval-

12 Oct 2026

Industry

Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash

We are expanding the Clef decision model family with Clef-omni, natively processing audio, video, images, and text in a single pipeline. We’ve also lowered Clef-flash pricing and boosted Clef inferenc

industrycloudflare-ai

9 Oct 2026

Reading the Whole Heart: Latent-Attention Masked Autoencoders for Multimodal Cardiac Representation Learning

arXiv:2609.12035v3 Announce Type: replace Abstract: Cardiovascular diagnosis and treatment rest on integrating complementary modalities, such as electrocardiogram, echocardiography, and chest X-rays,

9 Oct 2026

A method for multimodal analysis of TAIGA experiment data using essential features

arXiv:2610.08985v1 Announce Type: cross Abstract: The aim of processing and analyzing experimental data from physical experiments is to obtain physically significant information about the phenomenon u

researcharxiv-cs-lg

8 Oct 2026

MemPilot: Orchestrating On-Demand Multimodal Memory Curation for LLM Agents

arXiv:2610.06830v1 Announce Type: cross Abstract: Memory has become integral to the LLM agent ecosystem, supporting information retention and reuse across interactions. However, most existing agent me

safetyarxiv-cs-lg

6 Oct 2026

Structural-Functional Brain Connectivity Generation via Multimodal Hypergraph-based Flow Matching

arXiv:2610.02722v1 Announce Type: new Abstract: Structural connectivity (SC) and functional connectivity (FC) provide complementary information on interactions between brain regions and are widely use

researcharxiv-cs-lg

5 Oct 2026

Beyond Text: LLM-Based Dimensional Emotion Evaluation in Multimodal Dialogue

arXiv:2609.39072v1 Announce Type: cross Abstract: Emotion recognition in conversation has been widely studied, but applying Large Language Models (LLMs) to continuous dimensional emotion evaluation in

1 Oct 2026

Data-Driven Priors for Uncertainty-Aware Risk Prediction of Clinical Deterioration using Multimodal Data

arXiv:2603.08459v2 Announce Type: replace Abstract: Safe predictions are a crucial requirement for integrating predictive models into clinical decision support systems. One approach to improving trust

applicationsarxiv-cs-lg

1 Oct 2026

MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting

arXiv:2609.38523v1 Announce Type: cross Abstract: Financial forecasting from earnings conference calls requires models to reason over complex corporate disclosures, market expectations, and subtle com

1 Oct 2026

Agents

EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory

arXiv:2609.37923v1 Announce Type: new Abstract: Agents can learn from past executions, but enabling different agents to reuse and build on one another's experience remains challenging. We introduce Ep

agentsarxiv-cs-cv

Rethinking Multimodal Fake News Detection in the Generative AI Era

arXiv:2609.36850v1 Announce Type: new Abstract: Generative content is increasingly entering the production and dissemination of news, transforming fake news from manually fabricated or simply manipula

applicationsarxiv-cs-cl

Towards Breaking the Learning System Wall Using Multimodal Tutoring Transcriptions

arXiv:2609.36502v1 Announce Type: cross Abstract: Past research using log data has faced the 'learning system wall,' whereby few methods exist for generalizing models of student learning across platfo

VesselBench-800K: A Large-scale Perception Benchmark for Multimodal Vessel Detection, Counting, and Density Estimation

arXiv:2609.37003v1 Announce Type: new Abstract: Vessel perception from space is crucial for a wide range of maritime applications, from traffic monitoring to environmental protection. However, most ex

Visual sensitivity is not claim retractability: persistence-aware credit assignment for multimodal reinforcement learning

arXiv:2609.36572v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has been extended to Large Vision-Language Models (LVLMs), and perception-aware methods further en

Admissible Diffusion for Multimodal Interventional Trajectories

arXiv:2609.34433v1 Announce Type: new Abstract: Generating a plausible clinical trajectory does not establish what would happen under a different treatment. We present ADMIT, a framework combining irr

researcharxiv-cs-lg

Advancing Wildlife Conservation through Multimodal Animal Re-Identification with Environmental Metadata

arXiv:2609.34094v1 Announce Type: new Abstract: Identifying individual animals is crucial for effective wildlife monitoring and conservation efforts. Recent advancements in computer vision have shown

researcharxiv-cs-cv

Multimodal LLMs Outperform Pathology Foundation Models in Cross-Domain Histological Similarity

arXiv:2609.32876v1 Announce Type: new Abstract: State-of-the-art pathology foundation models, trained on millions of histology tiles, can fail to preserve tissue similarity when comparisons cross slid

SciGen-Verifier: A Multimodal Reasoner for Explainable Verification in Scientific Image Generation

arXiv:2609.33399v1 Announce Type: new Abstract: In realistic education, a solution is often expressed not only in words but in a drawing--a circuit, a geometric construction, a function plot--and a te

Agents

SegBanana: Steering Unified Multimodal Models into Medical Segmenters

arXiv:2609.34235v1 Announce Type: new Abstract: Medical image segmentation remains challenging in practical deployment, as models often struggle to generalize beyond the distributions covered by their

agentsarxiv-cs-cv

A Multimodal Dataset for Survival Prediction in Resected Pancreatic Ductal Adenocarcinoma

arXiv:2609.29726v1 Announce Type: new Abstract: Survival research in pancreatic ductal adenocarcinoma (PDAC) is limited by the scarcity of datasets linking whole-slide histology with clinical, molecul

researcharxiv-cs-cv

25 Sept 2026

C3M: Cross-Session Multimodal Memory Maintenance for Long-Horizon Tasks

arXiv:2609.29735v1 Announce Type: new Abstract: Long-horizon tasks require preserving and later recovering cross-session evidence under a bounded, query-blind memory budget. Existing compression can d

25 Sept 2026

Implicit Behavior Coordination from Sub-Task Demonstrations by Exploiting Overlap-Induced Multimodality

arXiv:2607.09234v2 Announce Type: replace Abstract: Long-horizon robotic rearrangement is commonly formulated as a skill-sequencing problem, where distinct behaviors are explicitly represented and coo

safetyarxiv-cs-ro

25 Sept 2026

A Leakage-Aware Multimodal Evaluation Framework for Early Intraoperative Acute Kidney Injury Prediction

arXiv:2609.26848v1 Announce Type: cross Abstract: Postoperative acute kidney injury (AKI) after major non-cardiac surgery carries substantial morbidity, yet early intraoperative risk stratification re

24 Sept 2026

PRISM-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Tobacco Product and Legislative Policy Reasoning

arXiv:2609.23769v1 Announce Type: new Abstract: The disambiguation of semantically similar statutory text across jurisdictions is a retrieval problem that existing methods do not solve. This inter-con

safetyarxiv-cs-cv

22 Sept 2026

Local Ai

RoboTalk: Learning Multi-Robot Communication and Coordination from Multimodal Demonstrations

arXiv:2609.23997v1 Announce Type: new Abstract: Multi-robot collaboration could enable more efficient and scalable solutions to complex robotic tasks, but collaboration under partial observability rem

local-aiarxiv-cs-ro

22 Sept 2026

TAC-Time: Texts as Channels For Multimodal Time Series Forecasting

arXiv:2609.24156v1 Announce Type: new Abstract: Most existing time series forecasting methods rely solely on numerical observations, overlooking rich contextual information from auxiliary texts. Recen

safetyarxiv-cs-cl

22 Sept 2026

E-AVI: Evidence-Grounded Multimodal Assessment for Automated Video Interviews

arXiv:2609.20001v1 Announce Type: new Abstract: Automated video interview assessment integrates verbal content, acoustic delivery, and visual behavior, yet numerical predictions alone provide limited

Integrating knowledge from case reports: a medical ontology based multimodal information system with structured summary

arXiv:2609.19775v1 Announce Type: new Abstract: Published medical case reports serve as a crucial medical information carrier, documenting discoveries in rare diseases, diagnostic methods, and innovat

Lens: Bringing the Right Semantic Perspective into Focus for Training-Free Multimodal Representation Learning

arXiv:2609.20252v1 Announce Type: cross Abstract: High-quality representations are essential for a wide range of downstream tasks. Dedicated embedding models are explicitly optimized for representatio

OHRID-Retail: An Open Multimodal Dataset of Human Activity in Retail Environments

arXiv:2609.19302v1 Announce Type: new Abstract: Open datasets describing human behavior in environments shared with mobile robots remain limited, particularly for retail activities that combine locomo

researcharxiv-cs-ro

To Memories and Beyond: From Remembering to Knowing You across Long-Term Multimodal Personal Archives

model-releasesarxiv-cs-cl

arXiv:2609.19167v1 Announce Type: new Abstract: As AI systems evolve into personalized digital companions, a central capability is reasoning over a user's long-term personal history: not merely storin

Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models

model-releasesarxiv-cs-cl

arXiv:2412.16555v4 Announce Type: replace Abstract: Large language models (LLMs) are widely applied in various fields of society due to their powerful reasoning, understanding, and generation capabili

17 Sept 2026

Multi-Session Multimodal Underwater Mapping with Acoustic and Optical Imaging

arXiv:2609.17929v1 Announce Type: new Abstract: Accurate seafloor mapping is essential for marine science, archaeology, and environmental monitoring. However, integrating data from different sensors,

safetyarxiv-cs-ro

17 Sept 2026

A multimodal large language model for evidence-based autism spectrum disorder screening

arXiv:2609.16464v1 Announce Type: new Abstract: The clinical management of autism spectrum disorder (ASD) faces a bottleneck in early screening, mainly because trained specialists are scarce and conve

researcharxiv-cs-cv

16 Sept 2026

A Vision-Language Foundation Model for Precise and Comprehensive Brain Tumor Diagnosis from Preoperative Multimodal Data

arXiv:2609.16597v1 Announce Type: cross Abstract: Background Non-invasive presurgical diagnosis of brain tumor types from Magnetic Resonance Imaging (MRI) is essential but challenging due to overlappi

applicationsarxiv-cs-ai

16 Sept 2026

MUUNRiver-Bench: Diagnosing Relation-Dependent Music Retrieval with Multimodal Instructions

arXiv:2609.16090v1 Announce Type: cross Abstract: Music retrieval is relation-dependent: given a reference track, a listener may seek its style with a new theme, a cover, or a comparable voice, and th