Search: “multimodal”

arXiv:2610.00623v1 Announce Type: new Abstract: Speculative decoding has achieved substantial lossless speedups for LLMs, but remains less effective for large vision-language models (LVLMs), where lig

tutorialsarxiv-cs-cv

2 Oct 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogies

DGX agent

arXiv:2610.01383v1 Announce Type: new Abstract: Analogical reasoning involves identifying and preserving relational structures across domains. However, existing approaches to AI-driven multimodal anal

model-releasesarxiv-cs-ai

2 Oct 2026

Can We Anticipate Violence? Multimodal Learning from Pre-Incident Behavioral Cues

arXiv:2609.40014v1 Announce Type: new Abstract: Detecting violence after it begins is important from recognizing behavioral cues that appear immediately beforehand. This work studies short-horizon pre

Industry

Cloudflare debuts open-weight multimodal decision models Clef and Clef-flash, claiming they are smarter and faster than Jev, based on Qwen3.8-27B and Qwen3.5-9B (Brandon Vigliarolo/The Register)

Brandon Vigliarolo / The Register: Cloudflare debuts open-weight multimodal decision models Clef and Clef-flash, claiming they are smarter and faster than Jev, based on Qwen3.8-27B and Qwen3.5-9B — Su

industrytechmeme

DecoMoE: Decoupling Visual Propagation and Expert Computation for Efficient Multimodal MoE Inference

arXiv:2609.38823v1 Announce Type: new Abstract: Multimodal mixture-of-experts (MoE) models combine sparse expert activation with visual-language capabilities, yet their inference remains costly becaus

Does Gradient Conflict Predict the Understanding--Generation Trade-off? A Controlled Audit of Conflict-Metric Validity in Unified Multimodal Models

arXiv:2609.38465v1 Announce Type: cross Abstract: Unified multimodal models (UMMs) are increasingly designed around gradient conflict between understanding and generation objectives. The premise that

researcharxiv-cs-ai

From Compound Figures to Medical Multi-image Reasoning: Scaling Multimodal Large Language Models with Biomedical Literature

arXiv:2511.22232v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) are increasingly capable in medical imaging, yet most focus on single-image settings. Clinical interp

From Image Interpretation to Clinical Reasoning: Upstream Physician-Context-Aware Multimodal Learning with Causal Reinforcement Learning

arXiv:2609.38924v1 Announce Type: new Abstract: Major adverse cardiovascular events (MACE) remain the leading cause of mortality worldwide. Opportunistic screening using routinely acquired clinical da

Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation

arXiv:2609.39120v1 Announce Type: new Abstract: On-policy distillation (OPD) improves reasoning by providing token-level supervision from a teacher on a student's own trajectories. Existing methods pr

Mutual Equilibrium: Multimodal Representation Learning through Reciprocal Feedback

arXiv:2609.39456v1 Announce Type: cross Abstract: This work proposes a mutual feedback architecture, MEQ, that refines the two inputs, of possibly different modalities, into a pair of coupled embeddin

Reinforcing Multimodal Reasoning via Token-Level Perception-Grounded Advantage Estimation

arXiv:2609.39168v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has improved the reasoning capabilities of Multimodal Large Language Models (MLLMs), yet existing

researcharxiv-cs-ai

MatToolBench: Benchmarking Multimodal Agents in Real-World Materials Science Workflows

arXiv:2609.37053v1 Announce Type: new Abstract: Multimodal GUI agents have achieved impressive results on general software benchmarks, yet their ability to operate professional scientific software rem

MoTIF-X: A Multimodal Tokenized Framework for Interpretable and Extensible Molecular Representation Learning

arXiv:2609.37384v1 Announce Type: new Abstract: Molecular representation learning is central to computer-aided drug discovery. Molecular graphs, SMILES strings, and 3D conformations provide complement

researcharxiv-cs-lg

Task-Oriented Visual Feature Compression via Residual Vector Quantization for Device-Edge Multimodal Inference

arXiv:2609.37090v1 Announce Type: new Abstract: Large multimodal models (LMMs) support diverse visual understanding and reasoning tasks but are often impractical to run entirely on resource-constraine

local-aiarxiv-cs-cv

VISTA: Internalizing Collective Visual Experience via On-Policy Distillation for Active Multimodal Agents

arXiv:2609.38086v1 Announce Type: new Abstract: Active multimodal agents use visual tools to acquire task-relevant evidence while reasoning. Although reinforcement learning samples multiple interactio

Agents

FlowAct-R2: Beyond Talking Avatar via Streaming Multimodal References and Proactive Agent Planning

arXiv:2609.35728v1 Announce Type: new Abstract: We present FlowAct-R2, a framework for interactive humanoid video generation that combines continuous multimodal control with proactive agent planning.

agentsarxiv-cs-cv

InfiMed2: A Generalist Medical Multimodal Foundation Model from Contextual Evidence and Stability-Aware Supervision

arXiv:2609.34798v1 Announce Type: cross Abstract: Recent medical multimodal models have benefited from larger corpora, broader modality coverage, and stronger reasoning-oriented training, yet effectiv

MAD-Guard: Controlled Study of Autoregressive Generation versus Direct Decision Interfaces for Closed Multimodal Forensic Tasks

arXiv:2609.33683v1 Announce Type: new Abstract: When should multimodal foundation models generate tokens, and when should they directly output a decision? We present MAD-Guard, a controlled study of o

Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment

arXiv:2609.35291v1 Announce Type: cross Abstract: Modern AI models are aligned through post-training to adapt them to downstream tasks. Recent work shows that fine-tuning language models on narrow tas

Applications

T-MoXAI: A Hierarchical Explainability Framework for Temporal Multimodal Data

arXiv:2609.33685v1 Announce Type: new Abstract: Artificial Intelligence (AI) models for temporal multimodal data have potential in healthcare and agriculture, but their opacity can limit trust and ado

applicationsarxiv-cs-lg

VaME: Exploring Variational Latent Reasoning for Multimodal Embeddings

model-releasesarxiv-cs-cv

arXiv:2609.33402v1 Announce Type: new Abstract: Universal multimodal retrieval requires compact embeddings that preserve task-relevant semantic information across diverse modalities. Prior works have

Hyperbolic Multimodal Continual Learning: A Closest-Admissible Solution

arXiv:2609.29329v1 Announce Type: cross Abstract: Existing continual-learning methods protect parameters, replayed examples, or Euclidean feature subspaces. When applied to hyperbolic multimodal model

25 Sept 2026

Long-Tail Adaptive Flow Matching with Explicit Conditional Consistency Guidance for Precise Multimodal Face Synthesis

arXiv:2609.29581v1 Announce Type: new Abstract: Although diffusion-based methods have substantially improved the controllability of multimodal face synthesis, their semantic alignment remains suboptim

25 Sept 2026

SemMSA: Latent Semantic-Aided Robust Multimodal Sentiment Analysis with Incomplete Data

arXiv:2609.30238v1 Announce Type: new Abstract: Recent research on Multimodal Sentiment Analysis (MSA) has focused on learning from language, visual, and acoustic modalities with incomplete data to in

safetyarxiv-cs-cl

25 Sept 2026

HINT-Blimp: Human INTent Inference from Multimodal Cues for Robotic Blimps

arXiv:2609.27154v1 Announce Type: new Abstract: In human-robot interaction, traditional interfaces such as joysticks and handheld tablets introduce latency into navigation tasks and require the operat

researcharxiv-cs-ro

LAYERSCOPE: A Layerwise Characterization of Video and Multimodal Learned Representations

arXiv:2609.28086v1 Announce Type: cross Abstract: We propose LAYERSCOPE, a label-free, layerwise framework that aims to characterize a model's learned representations in video and multimodal settings.

local-aiarxiv-cs-ai

Learning What to Activate: Combinatorial Capability Allocation for Long-Horizon Multimodal Agents

arXiv:2609.27869v1 Announce Type: new Abstract: Long-horizon multimodal agents rely on specialized capabilities for perception, retrieval, reasoning, verification, and execution. Existing designs typi

Small Cues, Big Consequences: Learning Pivotal Cues for Multimodal Meme Classification

model-releasesarxiv-cs-cl

arXiv:2609.26907v1 Announce Type: cross Abstract: Memes often derive their harmful, hateful, or sarcastic meaning from small but decisive visual, textual, or cross-modal cues. Existing multimodal clas

When Point Clouds Outperform Pixels: Rethinking Zero-Shot Multimodal Anomaly Detection

model-releasesarxiv-cs-cv

arXiv:2609.25793v1 Announce Type: new Abstract: Zero-shot multimodal anomaly detection commonly assumes that RGB and point cloud modalities are equally reliable and can contribute uniformly to anomaly

23 Sept 2026

Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey

arXiv:2507.22920v2 Announce Type: replace Abstract: The rapid advancement of large language models (LLMs) has intensified the need for effective mechanisms to transform continuous multimodal data into

safetyarxiv-cs-cl

22 Sept 2026

Agents

Towards robust multimodal 3D object detection via visual foundation models

arXiv:2609.23541v1 Announce Type: new Abstract: Multimodal 3D object detection is fundamental to robust perception in autonomous driving because it integrates complementary information from LiDAR and

agentsarxiv-cs-cv

22 Sept 2026

Explainable Multimodal Aspect-Based Sentiment Analysis with Dependency-guided Large Language Model

arXiv:2601.06848v2 Announce Type: replace Abstract: Multimodal aspect-based sentiment analysis (MABSA) aims to identify aspect-level sentiments by jointly modeling textual and visual information, whic

researcharxiv-cs-cl

From Stress to Affect: Multimodal Deep Learning for Physiological Emotion Recognition Across Wearable Sensor Modalities

arXiv:2609.20991v1 Announce Type: new Abstract: Physiological emotion recognition using wearable sensors has important applications in mental health monitoring, affective computing, and human-computer

researcharxiv-cs-lg

MIST: Multimodal Survival Prediction with Genomic-Guided Histology Attention

arXiv:2609.21811v1 Announce Type: new Abstract: Multimodal survival models can combine complementary prognostic information from whole-slide images and genomic profiles, but effective fusion remains c

Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction

model-releasesarxiv-cs-cl

arXiv:2609.21392v1 Announce Type: new Abstract: Natural audio-visual interaction is emerging as an important interface for AI assistants, allowing users to communicate through speech and vision rather

Efficient Unified Multimodal Understanding (EUMU): Winning Solution for the MUMU Track at the 8th LSVOS Challenge

arXiv:2609.19451v1 Announce Type: new Abstract: The Mobile Unified Multimodal Understanding (MUMU) Challenge requires a single efficient model to jointly perform multi-concept image tagging, open-voca

Fingerprinting Multimodal Large Language Models

arXiv:2609.20457v1 Announce Type: cross Abstract: While multimodal large language models (MLLMs) enable a wide range of image-text reasoning tasks, recent incidents indicate that they are vulnerable t

Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion

model-releasesarxiv-cs-cl

arXiv:2609.19417v1 Announce Type: new Abstract: Graph-based retrieval-augmented generation (RAG) is widely used for multimodal, cross-document question answering. However, building corpus-level graphs

QoS-Aware Federated Learning for Multimodal In-Cabin Interaction in Smart Vehicles

arXiv:2609.20123v1 Announce Type: new Abstract: Modern smart vehicles leverage multimodal sensors, ranging from high-bandwidth vision systems to low-rate physiological monitors, to provide personalize

local-aiarxiv-cs-lg

Anchoring What Matters: A Dual-Level Learning Framework for Visually-Grounded Multimodal Reasoning

arXiv:2609.18057v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has significantly improved the reasoning capabilities of large vision-language models (LVLMs). How

17 Sept 2026

LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition

arXiv:2510.08928v2 Announce Type: replace Abstract: Existing benchmarks for large multimodal models (LMMs) often fail to capture their performance in real-time, adversarial environments. We introduce

17 Sept 2026

Layers, Sinks, and Scaling: Adaptive Evidence Selection for Multimodal Large Language Models

arXiv:2609.16795v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can answer knowledge-intensive visual questions by combining visual evidence from images with facts retrieved f

16 Sept 2026

Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI

arXiv:2609.15888v1 Announce Type: cross Abstract: Deep networks trained on structural MRI for Alzheimer's disease (AD) staging often reach reasonable accuracy while attending to anatomically irrelevan

local-aiarxiv-cs-ai

Hypergraph-Regularized Gramian Volumes for Multimodal Retrieval

arXiv:2609.15320v1 Announce Type: new Abstract: Volume-based multimodal retrieval jointly scores a text query with a candidate's video, audio, and subtitle embeddings. While this approach captures hig

IMPACT-VLA: Interaction-aware Multimodal Propagation Attribution via Counterfactual Trajectories for Vision-Language-Action Policies

arXiv:2609.15005v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) policies perform robot manipulation tasks using multimodal inputs such as visual observations, proprioceptive states, and

Learning Multimodal One-step Flow Policy via Value-weighted Optimal Transport

arXiv:2609.15883v1 Announce Type: cross Abstract: Offline reinforcement learning aims to learn a policy solely from fixed datasets, which often contain multimodal action distributions. Flow policies c

MARCUS: An agentic, multimodal vision-language model for cardiac diagnosis and management

arXiv:2603.22179v2 Announce Type: replace Abstract: Cardiovascular disease remains the leading cause of global mortality, with progress hindered by human interpretation of complex cardiac tests. Curre

Not All Prompts Are Equal: Exploration-Guided Prompt Scaffolding for Multimodal Reinforcement Post-Training

arXiv:2609.15051v1 Announce Type: cross Abstract: Training prompts in online reinforcement learning (RL) differ substantially in how informative they are for the current policy: some are already satur