Search: “multimodal”

Benchmarking Multimodal Large Language Models for Zero-Shot Human-Object Interaction

arXiv:2610.18161v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong zero-shot reasoning across diverse domains, but their potential for Human-Object Inter

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

Focus on What You Need: Global-to-Local Dynamic Attention for Multimodal Sarcasm Detection

DGX agent

arXiv:2610.17707v1 Announce Type: new Abstract: Multimodal sarcasm detection has attracted increasing attention due to the widespread use of images and text on social media platforms. Existing studies

local-aiarxiv-cs-lg

14 Oct 2026

Prior-Fitted and Frozen Scorers for Multimodal Contrastive Learning

arXiv:2610.14770v1 Announce Type: new Abstract: Multimodal contrastive learning typically relies on manually specified scoring functions, such as cosine similarity, to quantify the similarity between

applicationsarxiv-cs-lg

Scaling Laws for Multimodal Data Mixtures

arXiv:2610.14423v1 Announce Type: cross Abstract: Frontier AI systems are increasingly natively multimodal, jointly pretrained on multiple modalities, such as text, vision, and audio. However, the sci

Where is the multimodal goal post? On the Ability of Foundation Models to Recognize Highlights in Videos

arXiv:2601.16333v3 Announce Type: replace Abstract: Foundation models are used for many real-world applications involving language understanding and generation from temporally-ordered multimodal event

applicationsarxiv-cs-cv

Tutorials

Building search across text, images, audio, and video? Multimodal search uses vector embeddings to retrieve content based on meaning, not ju…

Building search across text, images, audio, and video? Multimodal search uses vector embeddings to retrieve content based on meaning, not just keywords or metadata. Less manual tagging. More flexible

tutorialspinecone--x

13 Oct 2026

Evidential Multimodal Transformers with Dempster-Shafer Fusion for Emotion Recognition

arXiv:2610.13013v1 Announce Type: new Abstract: Softmax-based multimodal emotion recognition methods often assign high confidence even when sensor data are corrupted or contradictory, for example when

safetyarxiv-cs-lg

12 Oct 2026

MHR-Router: Multimodal Heterogeneous Retrieval-Enhanced Adaptive Routing for VLM Inference

arXiv:2610.13278v1 Announce Type: new Abstract: Vision-language models (VLMs) exhibit complementary capabilities, yet selecting the most suitable expert for a given image-question query remains challe

12 Oct 2026

Learning What to Trust in Multimodal Learning under Noisy Supervision

arXiv:2610.11057v1 Announce Type: new Abstract: Multimodal classification processes and relates information from multiple modalities to achieve more accurate predictions. However, existing methods typ

safetyarxiv-cs-cv

9 Oct 2026

Batch Augmentation with Unimodal Fine-tuning for Multimodal Fusion of Large Language Models

arXiv:2505.06592v2 Announce Type: replace Abstract: In this paper, we propose batch augmentation with unimodal fine-tuning for multimodal learning. We start with pre-trained unimodal models. We fine-t

researcharxiv-cs-cv

8 Oct 2026

Visual Abstention in Unified Multimodal Models

arXiv:2610.07887v1 Announce Type: cross Abstract: Unified multimodal models (UMMs) integrate understanding and generation, yet their generative behavior is rarely governed by what they understand abou

7 Oct 2026

Multimodal Deep Survival Analysis for Sinkhole Susceptibility

arXiv:2610.06365v1 Announce Type: new Abstract: Sinkholes are a widespread geohazard in karst terrain. In Florida, soluble carbonate bedrock, shallow groundwater, and intense rainfall combine to make

applicationsarxiv-cs-lg

6 Oct 2026

MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoning

arXiv:2610.01352v1 Announce Type: new Abstract: Open multimodal reasoning models have benefited from large-scale reasoning supervision, yet reliable post-training remains challenging due to uneven dat

2 Oct 2026

Walking the Embedding Space: Datastore Extraction from Multimodal RAG

arXiv:2610.01871v1 Announce Type: cross Abstract: Multimodal Retrieval-Augmented Generation (MRAG) has emerged as a reliable and cost-effective technique of grounding the generative capabilities of Mu

applicationsarxiv-cs-ai

2 Oct 2026

Beyond Layers: Position-Resolved Gradient Conflict and Position-Aware Modulation for Unified Multimodal Models

arXiv:2609.38485v1 Announce Type: new Abstract: Unified multimodal models (UMMs) train image understanding and autoregressive image generation on shared parameters, and the two objectives are known to

1 Oct 2026

ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge

arXiv:2602.09839v2 Announce Type: replace Abstract: Existing multimodal retrieval benchmarks largely emphasize semantic matching on daily-life images and offer limited diagnostics of professional know

30 Sept 2026

Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment

arXiv:2501.18157v2 Announce Type: replace-cross Abstract: Building reliable speech systems often requires combining multiple modalities, like audio and visual cues. While such multimodal solutions fre

applicationsarxiv-cs-cv

30 Sept 2026

Agents

MERID: Multimodal Exploration via Recursive Self-Improvement Agents for Major Depression Analysis

arXiv:2609.36235v1 Announce Type: new Abstract: Major depressive disorder (MDD) severely impacts daily activities and quality of life. Detecting MDD involves multimodal data, such as interview recordi

agentsarxiv-cs-ai

30 Sept 2026

BioEVAL: A global, multi-institutional benchmark of large language and multimodal models for bioengineering

arXiv:2609.30489v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated historic breakthroughs in general reasoning with early successes in biomedical science. However, existing

Can Linguistic Reasoning Vectors Enhance Multimodal Reasoning Ability?

arXiv:2609.31140v1 Announce Type: new Abstract: Most Vision-Language Models (VLMs) are built by extending pretrained Large Language Models (LLMs) with visual modules and multimodal alignment. However,

Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward

arXiv:2511.20561v3 Announce Type: replace-cross Abstract: Recent years have witnessed significant progress in Unified Multimodal Models, yet a fundamental question remains: Does understanding truly in

researcharxiv-cs-cl

Improving Visual Sensitivity of LLMs on Multimodal Machine Translation with Metric-based Loss Weighting

arXiv:2609.31169v1 Announce Type: cross Abstract: Multimodal Machine Translation aims to incorporate additional signal from non-textual modalities to improve translations by resolving ambiguities. Whi

What Improves Multimodal Misinformation Detection? Answers from a Large-Scale Empirical Study

arXiv:2609.30402v1 Announce Type: cross Abstract: Multimodal misinformation is increasingly crafted to look convincing by pairing a textual claim with an image that appears to 'prove' it. Yet in pract

M^2PFN: End-to-End Disentangled Alignment for Generalizable Multimodal In-Context Learning in Alzheimer's Disease

arXiv:2609.28836v1 Announce Type: cross Abstract: While various multimodal methods combining imaging and tabular data for Alzheimer's disease (AD) diagnosis were proposed, they are often limited in ge

safetyarxiv-cs-ai

25 Sept 2026

Confidence Falls Short: Asymmetric Certainty Gains from Optimization Hinder Multimodal Classification

arXiv:2609.28165v1 Announce Type: new Abstract: Multimodal learning (MML) falls into the optimization dilemma due to the modality imbalance phenomenon, leading to suboptimal overall performance in pra

24 Sept 2026

Probabilistic Modeling of Jailbreak on Multimodal LLMs: From Quantification to Application

arXiv:2503.06989v5 Announce Type: replace-cross Abstract: Recently, Multimodal Large Language Models (MLLMs) have demonstrated their superior ability in understanding multimodal content. However, they

23 Sept 2026

Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning

arXiv:2506.07044v5 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in understanding common visual elements, largely due to their lar

researcharxiv-cs-cl

22 Sept 2026

MMS-VPR: A Fine-Grained Multimodal Street-Level Visual Place Recognition Dataset and Evaluation Benchmark for Dense Pedestrian Environments

arXiv:2505.12254v3 Announce Type: replace Abstract: Existing visual place recognition (VPR) datasets predominantly rely on vehicle-mounted imagery, offer limited multimodal diversity, and underreprese

22 Sept 2026

Generalist-Specialist Mixture-of-Experts for Rare Pathology Detection in Multimodal Imaging

arXiv:2609.18688v1 Announce Type: cross Abstract: AI models for multimodal medical imaging must balance modality-specific specialization with cross-modal shared representations, a trade-off that pure

17 Sept 2026

MINT: Multimodal Imaging-to-Speech Knowledge Transfer for Early Alzheimer's Screening

arXiv:2602.23994v2 Announce Type: replace-cross Abstract: Alzheimer's disease is a progressive neurodegenerative disorder in which mild cognitive impairment (MCI) precedes dementia. Structural MRI pro

17 Sept 2026

Transcribe, Then Reason: Two-Pass Decomposition for Multimodal Review

arXiv:2609.18958v1 Announce Type: cross Abstract: The natural way to review a long recording or document with a multimodal model is to hand it the raw source and ask for a review in one call. We show

17 Sept 2026

SAVOR: Self-Aware Visual Grounding via Confidence-Calibrated Reinforcement Learning for Multimodal Hallucination Mitigation

arXiv:2609.16601v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have made strong progress on visual question answering and image captioning, yet they still produce fluent clai

safetyarxiv-cs-cv

16 Sept 2026

From Density to Biopsy Decisions and Malignancy Prediction: A Benchmark Study of Multimodal Large Language Models Against Radiologists in Digital and Contrast-Enhanced Mammography

arXiv:2609.14676v1 Announce Type: new Abstract: Purpose: To compare four multimodal large language models (MLLMs) with radiologists of varying expertise in breast density assessment, BI-RADS assessmen

15 Sept 2026

Reason What Matters: Retrieval-Grounded Reasoning for Universal Multimodal Embeddings

arXiv:2609.15296v1 Announce Type: new Abstract: Universal multimodal embedding (UME) learns unified representations across modalities, enabling a single model to support diverse retrieval tasks. Recen

safetyarxiv-cs-ai

15 Sept 2026

Local Ai

M3-Former: Multimodal Transformer with Mixture-of-Experts for Long-Term Vessel Trajectory Prediction

arXiv:2609.10559v1 Announce Type: cross Abstract: To address the challenges of behavioral multimodality, limited semantic utilization, and long-term error accumulation in vessel trajectory prediction,

local-aiarxiv-cs-cv

11 Sept 2026

CHIMERA Challenge Task 2 and 3: Response Subtypes Classification and Progression Survival Prediction in Bladder Cancer Patients using Multimodal Datasets

arXiv:2609.09510v1 Announce Type: cross Abstract: High-risk non-muscle-invasive bladder cancer (HR-NMIBC) carries substantial risks of recurrence and progression, while current clinical risk stratific

10 Sept 2026

Fine-Grained Visual Preprocessing and Dual-Stream Temporal Modeling for Multimodal Sentiment Analysis on Social Media

arXiv:2609.07010v1 Announce Type: new Abstract: Multimodal sentiment analysis often remains text-dominant due to raw-video noise and insufficient temporal modeling. Using CH-SIMS v2.0S, this study pro

researcharxiv-cs-cv

9 Sept 2026

MIDR: Enrichment-Augmented Indexing for Multimodal Document Retrieval

arXiv:2609.01316v1 Announce Type: cross Abstract: Retrieval over visually rich documents has a representation problem: important content often lives in tables, charts, figures, and layout relations th

2 Sept 2026

SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models

arXiv:2609.01004v1 Announce Type: cross Abstract: Despite their strong multimodal understanding ability, multimodal large language models (MLLMs) incur substantial computational overhead when processi

2 Sept 2026

Towards reliable multimodal disaster severity assessment through preference optimization and explainable vision-language reasoning

arXiv:2609.00879v1 Announce Type: new Abstract: Reliable disaster damage assessment requires models that provide both accurate predictions and transparent explanations. However, existing multimodal ap

safetyarxiv-cs-ai

2 Sept 2026

Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding

arXiv:2505.16990v3 Announce Type: replace Abstract: In this work, we propose Dimple, the first Discrete Diffusion Multimodal Large Language Model (DMLLM). We observe that training with a purely discre

Foundation and Multimodal Large Language Models for Face Presentation and Morph Attack Detection

arXiv:2608.29802v1 Announce Type: new Abstract: Face recognition systems are increasingly deployed in security-critical applications, yet they remain vulnerable to presentation and morph attacks. Pres

SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models

arXiv:2608.29098v1 Announce Type: new Abstract: Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior. Existing safeguards, howeve

Agents

VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs

arXiv:2608.30705v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) struggle with fine-grained Visual Search, the task of locating small or rare objects in high-resolution images.

agentsarxiv-cs-cv

A Multimodal Dataset of Student Oral Presentations with Sensors and Evaluation Data

arXiv:2601.07576v2 Announce Type: replace-cross Abstract: Oral presentation skills are a critical component of higher education, yet comprehensive datasets capturing real-world student performance acr

31 Aug 2026

Mitigating Strong-Modality Collapse in Multimodal Learning via Inverted Asymmetric Fusion

arXiv:2608.26879v1 Announce Type: new Abstract: Fusing multiple modalities is expected to improve model performance. However, on the MultiHuSE dataset, early, late, and symmetric attention fusion ofte

28 Aug 2026

MM-Spectrum: Multimodal Multi-spectral Molecular Structural Elucidation with a Stable MoE Framework

arXiv:2608.27286v1 Announce Type: new Abstract: Inferring molecular structures from multimodal spectroscopic measurements requires integrating complementary yet highly heterogeneous signals. However,

28 Aug 2026

Structured Evidence Routing for Incident Risk Prediction from Multimodal Longitudinal EHRs

arXiv:2608.26191v1 Announce Type: new Abstract: Incident risk prediction from longitudinal electronic health records (EHRs) is challenging because relevant signals are multimodal, weak in isolation, a