Search: “multimodal”

Learning What to Activate: Combinatorial Capability Allocation for Long-Horizon Multimodal Agents

arXiv:2609.27869v1 Announce Type: new Abstract: Long-horizon multimodal agents rely on specialized capabilities for perception, retrieval, reasoning, verification, and execution. Existing designs typi

24 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Small Cues, Big Consequences: Learning Pivotal Cues for Multimodal Meme Classification

DGX agent

arXiv:2609.26907v1 Announce Type: cross Abstract: Memes often derive their harmful, hateful, or sarcastic meaning from small but decisive visual, textual, or cross-modal cues. Existing multimodal clas

model-releasesarxiv-cs-cl

24 Sept 2026

When Point Clouds Outperform Pixels: Rethinking Zero-Shot Multimodal Anomaly Detection

arXiv:2609.25793v1 Announce Type: new Abstract: Zero-shot multimodal anomaly detection commonly assumes that RGB and point cloud modalities are equally reliable and can contribute uniformly to anomaly

23 Sept 2026

Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey

arXiv:2507.22920v2 Announce Type: replace Abstract: The rapid advancement of large language models (LLMs) has intensified the need for effective mechanisms to transform continuous multimodal data into

safetyarxiv-cs-cl

22 Sept 2026

Agents

Towards robust multimodal 3D object detection via visual foundation models

arXiv:2609.23541v1 Announce Type: new Abstract: Multimodal 3D object detection is fundamental to robust perception in autonomous driving because it integrates complementary information from LiDAR and

agentsarxiv-cs-cv

22 Sept 2026

Explainable Multimodal Aspect-Based Sentiment Analysis with Dependency-guided Large Language Model

arXiv:2601.06848v2 Announce Type: replace Abstract: Multimodal aspect-based sentiment analysis (MABSA) aims to identify aspect-level sentiments by jointly modeling textual and visual information, whic

researcharxiv-cs-cl

From Stress to Affect: Multimodal Deep Learning for Physiological Emotion Recognition Across Wearable Sensor Modalities

arXiv:2609.20991v1 Announce Type: new Abstract: Physiological emotion recognition using wearable sensors has important applications in mental health monitoring, affective computing, and human-computer

researcharxiv-cs-lg

MIST: Multimodal Survival Prediction with Genomic-Guided Histology Attention

arXiv:2609.21811v1 Announce Type: new Abstract: Multimodal survival models can combine complementary prognostic information from whole-slide images and genomic profiles, but effective fusion remains c

Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction

arXiv:2609.21392v1 Announce Type: new Abstract: Natural audio-visual interaction is emerging as an important interface for AI assistants, allowing users to communicate through speech and vision rather

Efficient Unified Multimodal Understanding (EUMU): Winning Solution for the MUMU Track at the 8th LSVOS Challenge

arXiv:2609.19451v1 Announce Type: new Abstract: The Mobile Unified Multimodal Understanding (MUMU) Challenge requires a single efficient model to jointly perform multi-concept image tagging, open-voca

Fingerprinting Multimodal Large Language Models

arXiv:2609.20457v1 Announce Type: cross Abstract: While multimodal large language models (MLLMs) enable a wide range of image-text reasoning tasks, recent incidents indicate that they are vulnerable t

Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion

arXiv:2609.19417v1 Announce Type: new Abstract: Graph-based retrieval-augmented generation (RAG) is widely used for multimodal, cross-document question answering. However, building corpus-level graphs

Local Ai

QoS-Aware Federated Learning for Multimodal In-Cabin Interaction in Smart Vehicles

arXiv:2609.20123v1 Announce Type: new Abstract: Modern smart vehicles leverage multimodal sensors, ranging from high-bandwidth vision systems to low-rate physiological monitors, to provide personalize

local-aiarxiv-cs-lg

Anchoring What Matters: A Dual-Level Learning Framework for Visually-Grounded Multimodal Reasoning

arXiv:2609.18057v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has significantly improved the reasoning capabilities of large vision-language models (LVLMs). How

17 Sept 2026

LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition

arXiv:2510.08928v2 Announce Type: replace Abstract: Existing benchmarks for large multimodal models (LMMs) often fail to capture their performance in real-time, adversarial environments. We introduce

17 Sept 2026

Layers, Sinks, and Scaling: Adaptive Evidence Selection for Multimodal Large Language Models

arXiv:2609.16795v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can answer knowledge-intensive visual questions by combining visual evidence from images with facts retrieved f

16 Sept 2026

Local Ai

Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI

arXiv:2609.15888v1 Announce Type: cross Abstract: Deep networks trained on structural MRI for Alzheimer's disease (AD) staging often reach reasonable accuracy while attending to anatomically irrelevan

local-aiarxiv-cs-ai

Hypergraph-Regularized Gramian Volumes for Multimodal Retrieval

arXiv:2609.15320v1 Announce Type: new Abstract: Volume-based multimodal retrieval jointly scores a text query with a candidate's video, audio, and subtitle embeddings. While this approach captures hig

IMPACT-VLA: Interaction-aware Multimodal Propagation Attribution via Counterfactual Trajectories for Vision-Language-Action Policies

arXiv:2609.15005v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) policies perform robot manipulation tasks using multimodal inputs such as visual observations, proprioceptive states, and

Learning Multimodal One-step Flow Policy via Value-weighted Optimal Transport

arXiv:2609.15883v1 Announce Type: cross Abstract: Offline reinforcement learning aims to learn a policy solely from fixed datasets, which often contain multimodal action distributions. Flow policies c

MARCUS: An agentic, multimodal vision-language model for cardiac diagnosis and management

arXiv:2603.22179v2 Announce Type: replace Abstract: Cardiovascular disease remains the leading cause of global mortality, with progress hindered by human interpretation of complex cardiac tests. Curre

Not All Prompts Are Equal: Exploration-Guided Prompt Scaffolding for Multimodal Reinforcement Post-Training

arXiv:2609.15051v1 Announce Type: cross Abstract: Training prompts in online reinforcement learning (RL) differ substantially in how informative they are for the current policy: some are already satur

PRI-Net: A Lightweight Multimodal Framework for 3D UAV Localization

arXiv:2609.14469v1 Announce Type: new Abstract: Accurate 3D localization of unmanned aerial vehicles (UAVs) remains challenging for existing multimodal approaches due to sparse LiDAR geometry, modalit

Agents

Sensory Precision Inference for Multimodal Arbitration under Uncertainty

arXiv:2609.15065v1 Announce Type: new Abstract: Autonomous agents operating on multisensory data cannot assume that all sensory modalities remain consistently informative. In real environments, sensor

agentsarxiv-cs-lg

Agents

Speak to the City: Multimodal Resolution for Outside-the-Vehicle References

arXiv:2609.14691v1 Announce Type: cross Abstract: As autonomous vehicles and Extended Reality (XR) headsets enable novel in-car interactions, seamlessly querying physical landmarks, known as Outside-t

agentsarxiv-cs-cl

Beyond the Query: Do Retrieval Signals Improve Adaptive Multimodal RAG Routing?

arXiv:2609.12437v1 Announce Type: new Abstract: Adaptive RAG often uses retrieval-time signals to decide whether another retrieval, reranking, or multimodal step should run. We ask whether these signa

researcharxiv-cs-lg

Applications

Detecting and Explaining Fake News Short Videos with Multimodal Content and Real-World Evidence

arXiv:2609.12678v1 Announce Type: new Abstract: Short-video platforms have become a primary news source for the public, which has also enabled the widespread dissemination of fake news videos. We stud

applicationsarxiv-cs-cv

Learning Sparse Latent Predictive Foundation Model for Multimodal Neuroimaging

arXiv:2606.14957v4 Announce Type: replace Abstract: Brain MRIs are routinely acquired as multiple complementary sequences with unique contrast weighting, including T1-weighed imaging (T1w) anatomic an

RoES: Rotational Equivariant Selective-frequency Fusion for Multimodal Images

arXiv:2609.12497v1 Announce Type: new Abstract: Infrared-visible image fusion facilitates robust multimodal perception by integrating complementary textural nuances from visible sensors with thermal s

Local Ai

Exploring Multimodal Prompt for Visualization Authoring with Large Language Models

arXiv:2504.13700v2 Announce Type: replace-cross Abstract: Recent advances in large language models (LLMs) have shown great potential in automating the process of visualization authoring through simple

local-aiarxiv-cs-ai

12 Sept 2026

Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents

arXiv:2609.11243v1 Announce Type: new Abstract: Autonomous research agents are increasingly expected to search the literature, analyze experimental evidence, and generate scientific hypotheses. These

12 Sept 2026

When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting

arXiv:2609.11282v1 Announce Type: new Abstract: Multimodal forecasting models that combine time series with text annotations promise richer prediction through textual context, but how do we know wheth

12 Sept 2026

BanglaMemeX: Advancing Cultural Metaphoric Image Interpretation in Bangla with a Multimodal Explainable Dataset

arXiv:2609.08029v1 Announce Type: cross Abstract: Vision Language Models have achieved strong performance on multimodal benchmarks, yet their ability to reason about culturally grounded and metaphor-r

GeoHeight-Bench: Towards Height-Aware Multimodal Reasoning in Remote Sensing

arXiv:2603.25565v2 Announce Type: replace Abstract: Current Large Multimodal Models (LMMs) in Earth observation are predominantly evaluated on planar optical tasks and often neglect the vertical dimen

Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Models

arXiv:2509.19994v3 Announce Type: replace Abstract: Multimodal pre-trained models (e.g., ImageBind), which align distinct data modalities into a shared embedding space, have shown remarkable success a

NOAH: Learning the Full Patient Journey. A Longitudinal Multimodal Time-Aware Model for Representation and Forecasting

arXiv:2609.09140v1 Announce Type: new Abstract: The digitization of healthcare has generated vast, longitudinal, and multimodal patient records over a lifetime, yet fully exploiting these data to repr

safetyarxiv-cs-lg

Radiation, Rotation and Scale Invariant Feature Descriptor for Multimodal Image Matching

arXiv:2609.06343v1 Announce Type: new Abstract: Multimodal image matching is a fundamental task for multi-source information fusion. However, geometric distortions and nonlinear radiometric difference

RAH-VLA: Resolution-Adaptive Hierarchical Vision-Language Alignment for Multimodal Remote Sensing Understanding

arXiv:2512.23243v3 Announce Type: replace Abstract: Multimodal vision-language modeling has emerged as a promising paradigm for remote sensing (RS) image understanding. However, existing methods are l

SAFIRE: Safety-Critical Benchmark for Fine-grained Fire and Smoke Understanding in Multimodal LLMs

arXiv:2609.07823v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) show strong progress on vision-language tasks, yet their reliability in safety-critical settings remains undere

Towards Unified Multimodal Graph Foundation Model: A Bridge-Router-Adapter Based Approach

arXiv:2609.06668v1 Announce Type: new Abstract: Multimodal graphs couple node attributes in different modalities, such as text and images, with relational structure, enabling topological structure and

researcharxiv-cs-lg

ICM-Bench: Person-Level Identity Reasoning in Multimodal Agents with Long-Term Memory

arXiv:2609.04438v1 Announce Type: new Abstract: Long-horizon multimodal agents should remember not only what happened but also who participated. This capability depends on linking recurring faces, voi

7 Sept 2026

MURAL: Multimodal Uncertainty-aware Recommendation via Adaptive edge Learning

arXiv:2609.04574v1 Announce Type: cross Abstract: Multimodal Graph Neural Networks have become standard for recommendation by augmenting sparse interaction data with content features. Yet current arch

safetyarxiv-cs-lg

7 Sept 2026

CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning

arXiv:2609.03526v1 Announce Type: new Abstract: Multimodal language models achieve near-ceiling scores on food recognition benchmarks, yet it remains unclear whether this success reflects genuine cult

4 Sept 2026

GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs

arXiv:2609.03892v1 Announce Type: cross Abstract: 3D spatial reasoning underpins understanding and acting in the physical world, yet it remains unreliable in current multimodal large language models (

researcharxiv-cs-ai

4 Sept 2026

MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models

arXiv:2609.01772v1 Announce Type: new Abstract: Meme understanding goes beyond recognizing visual content or literal text; it requires implicit cultural knowledge and pragmatic inference that most vis

3 Sept 2026

Multimodal Language Models as Text-to-Image Model Evaluators

arXiv:2505.00759v3 Announce Type: replace-cross Abstract: The steady improvements of text-to-image (T2I) generative models lead to slow deprecation of automatic evaluation benchmarks that rely on stat

3 Sept 2026

CoMMET: A Psychologically Grounded Benchmark for Evaluating Theory of Mind in Multimodal LLMs

arXiv:2603.11915v2 Announce Type: replace Abstract: Theory of Mind (ToM)-the ability to reason about the mental states of oneself and others-is a cornerstone of human social intelligence. As Multimoda

2 Sept 2026

Same Semantics, Different Outcome: On the Modality Robustness of Multimodal LLMs under Knowledge Conflict