Search: “multimodal”

On the Effectiveness of Integration Methods for Multimodal Dialogue Response Retrieval

arXiv:2506.11499v2 Announce Type: replace Abstract: Multimodal chatbots have become one of the major topics for dialogue systems in both research community and industry. Recently, researchers have she

5 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

PC-MNet: Dual-Level Congruity Modeling for Multimodal Sarcasm Detection via Polarity-Modulated Attention

DGX agent

arXiv:2605.02447v1 Announce Type: new Abstract: Multimodal sarcasm detection, which aims to precisely identify pragmatic incongruities between literal text and nonverbal cues, has gained substantial a

model-releasesarxiv-cs-cl

5 May 2026

UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors

arXiv:2605.00658v1 Announce Type: new Abstract: Recent progress has shown that video diffusion models (VDMs) can be repurposed for diverse multimodal graphics tasks. However, existing methods often tr

safetyarxiv-cs-cv

4 May 2026

PRISM: Pre-alignment via Black-box On-policy Distillation for Multimodal Reinforcement Learning

model-releasesarxiv-cs-ai

arXiv:2604.28123v1 Announce Type: cross Abstract: The standard post-training recipe for large multimodal models (LMMs) applies supervised fine-tuning (SFT) on curated demonstrations followed by reinfo

1 May 2026

Contrastive Learning for Multimodal Human Activity Recognition with Limited Labeled Data

arXiv:2604.23281v1 Announce Type: cross Abstract: Human activity recognition serves as the foundation for various emerging applications. In recent years, researchers have used collaborative sensing of

applicationsarxiv-cs-cv

28 Apr 2026

EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Models

model-releasesarxiv-cs-ai

arXiv:2502.04424v4 Announce Type: replace-cross Abstract: With the integration of multimodal large language models (MLLMs) into robotic systems and AI applications, embedding emotional intelligence (E

28 Apr 2026

MIMIC: A Generative Multimodal Foundation Model for Biomolecules

arXiv:2604.24506v1 Announce Type: new Abstract: Biological function emerges from coupled constraints across sequence, structure, regulation, evolution, and cellular context, yet most foundation models

28 Apr 2026

Agents

DM^3-Nav: Decentralized Multi-Agent Multimodal Multi-Object Semantic Navigation

arXiv:2604.22014v1 Announce Type: cross Abstract: We present DM^3-Nav, a fully decentralized multi-agent semantic navigation system supporting multimodal open-vocabulary goal specification and multi-o

agentsarxiv-cs-ro

Focus Session: Hardware and Software Techniques for Accelerating Multimodal Foundation Models

arXiv:2604.21952v1 Announce Type: cross Abstract: This work presents a multi-layered methodology for efficiently accelerating multimodal foundation models (MFMs). It combines hardware and software co-

Hardware

Multimodal Neural Operators for Real-Time Biomechanical Modelling of Traumatic Brain Injury

arXiv:2510.03248v3 Announce Type: replace-cross Abstract: Background: Traumatic brain injury modeling requires integrating volumetric neuroimaging, demographic parameters, and acquisition metadata. Fi

hardwarearxiv-cs-ai

V-MAGE: A Game Evaluation Framework for Assessing Vision-Centric Capabilities in Multimodal Large Language Models

arXiv:2504.06148v3 Announce Type: replace Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in visual-text processing. However, existi

researcharxiv-cs-cv

Latent Denoising Improves Visual Alignment in Large Multimodal Models

arXiv:2604.21343v1 Announce Type: new Abstract: Large Multimodal Models (LMMs) such as LLaVA are typically trained with an autoregressive language modeling objective, providing only indirect supervisi

24 Apr 2026

Tutorials

S1-VL: Scientific Multimodal Reasoning Model with Thinking-with-Images

arXiv:2604.21409v1 Announce Type: new Abstract: We present S1-VL, a multimodal reasoning model for scientific domains that natively supports two complementary reasoning paradigms: Scientific Reasoning

tutorialsarxiv-cs-cv

24 Apr 2026

MATT-Diff: Multimodal Active Target Tracking by Diffusion Policy

arXiv:2511.11931v2 Announce Type: replace Abstract: This paper proposes MATT-Diff: Multimodal Active Target Tracking by Diffusion Policy, a control policy for active multi-target tracking using a mobi

safetyarxiv-cs-ro

23 Apr 2026

Text Steganography with Dynamic Codebook and Multimodal Large Language Model

arXiv:2604.20269v1 Announce Type: cross Abstract: With the popularity of the large language models (LLMs), text steganography has achieved remarkable performance. However, existing methods still have

23 Apr 2026

How to Teach Large Multimodal Models New Skills

arXiv:2510.08564v2 Announce Type: replace Abstract: How can we teach large multimodal models (LMMs) new skills without erasing prior abilities? We study sequential fine-tuning on five target skills wh

safetyarxiv-cs-ai

22 Apr 2026

Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge

arXiv:2603.11665v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have been widely adopted as MLLM-as-a-Judges due to their strong alignment with human judgment across vario

safetyarxiv-cs-cl

22 Apr 2026

Seeing Candidates at Scale: Multimodal LLMs for Visual Political Communication on Instagram

arXiv:2604.19489v1 Announce Type: new Abstract: This paper presents a computational case study that evaluates the capabilities of specialized machine learning models and emerging multimodal large lang

applicationsarxiv-cs-cv

22 Apr 2026

Beyond Feature Fusion: Contextual Bayesian PEFT for Multimodal Uncertainty Estimation

model-releasesarxiv-cs-lg

arXiv:2604.16615v1 Announce Type: new Abstract: We introduce CoCo-LoRA, a multimodal, uncertainty-aware parameter-efficient fine-tuning method for text prediction tasks accompanied by audio context. E

CFMS: Towards Explainable and Fine-Grained Chinese Multimodal Sarcasm Detection Benchmark

arXiv:2604.16372v1 Announce Type: new Abstract: Multimodal sarcasm detection has recently garnered significant attention. However, existing benchmarks suffer from coarse-grained annotations and limite

Cross-Modal Bayesian Low-Rank Adaptation for Uncertainty-Aware Multimodal Learning

model-releasesarxiv-cs-lg

arXiv:2604.16657v1 Announce Type: new Abstract: Large pre-trained language models are increasingly adapted to downstream tasks using parameter-efficient fine-tuning (PEFT), but existing PEFT methods a

ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection

arXiv:2410.04509v3 Announce Type: replace Abstract: As the field of Multimodal Large Language Models (MLLMs) continues to evolve, their potential to revolutionize artificial intelligence is particular

Tutorials

Learning Invariant Modality Representation for Robust Multimodal Learning from a Causal Inference Perspective

arXiv:2604.18460v1 Announce Type: new Abstract: Multimodal affective computing aims to predict humans' sentiment, emotion, intention, and opinion using language, acoustic, and visual modalities. Howev

tutorialsarxiv-cs-lg

Multiplication in Multimodal LLMs: Computation with Text, Image, and Audio Inputs

arXiv:2604.18203v1 Announce Type: new Abstract: Multimodal LLMs can accurately perceive numerical content across modalities yet fail to perform exact multi-digit multiplication when the identical unde

Penny Wise, Pixel Foolish: Bypassing Price Constraints in Multimodal Agents via Visual Adversarial Perturbations

arXiv:2604.16515v1 Announce Type: new Abstract: The rapid proliferation of Multimodal Large Language Models (MLLMs) has enabled mobile agents to execute high-stakes financial transactions, but their a

RA-RRG: Multimodal Retrieval-Augmented Radiology Report Generation with Key Phrase Extraction

arXiv:2504.07415v2 Announce Type: replace-cross Abstract: Automated radiology report generation (RRG) holds potential to reduce the workload of radiologists, and recent advances in multimodal large la

applicationsarxiv-cs-cl

ReFineVLA: Multimodal Reasoning-Aware Generalist Robotic Policies via Teacher-Guided Fine-Tuning

arXiv:2604.17800v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have gained much attention from the research community thanks to their strength in translating multimodal observat

Using Perspectival Words Is Harder Than Vocabulary Words for Humans and Even More So for Multimodal Language Models

arXiv:2506.00065v2 Announce Type: replace Abstract: Multimodal language models (MLMs) increasingly demonstrate human-like communication, yet their use of everyday perspectival words remains poorly und

researcharxiv-cs-cl

Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs

arXiv:2604.16060v1 Announce Type: cross Abstract: Multimodal Reasoning Models (MRMs) leveraging Chain-of-Thought (CoT) based thinking have revolutionized mathematical and logical problem-solving. Howe

20 Apr 2026

Local Ai

GIST: Multimodal Knowledge Extraction and Spatial Grounding via Intelligent Semantic Topology

arXiv:2604.15495v1 Announce Type: new Abstract: Navigating complex, densely packed environments like retail stores, warehouses, and hospitals poses a significant spatial grounding challenge for humans

local-aiarxiv-cs-ai

20 Apr 2026

Knowing When Not to Answer: Evaluating Abstention in Multimodal Reasoning Systems

arXiv:2604.14799v1 Announce Type: new Abstract: Effective abstention (EA), recognizing evidence insufficiency and refraining from answering, is critical for reliable multimodal systems. Yet existing e

17 Apr 2026

MixAtlas: Uncertainty-aware Data Mixture Optimization for Multimodal LLM Midtraining

arXiv:2604.14198v1 Announce Type: cross Abstract: Domain reweighting can improve sample efficiency and downstream generalization, but data-mixture optimization for multimodal midtraining remains large

17 Apr 2026

Local Ai

A Domain-Specific Language for LLM-Driven Trigger Generation in Multimodal Data Collection

arXiv:2604.13046v1 Announce Type: cross Abstract: Data-driven systems depend on task-relevant data, yet data collection pipelines remain passive and indiscriminate. Continuous logging of multimodal se

local-aiarxiv-cs-cl

Dental-TriageBench: Benchmarking Multimodal Reasoning for Hierarchical Dental Triage

arXiv:2604.13060v1 Announce Type: new Abstract: Dental triage is a safety-critical clinical routing task that requires integrating multimodal clinical information (e.g., patient complaints and radiogr

Indexing Multimodal Language Models for Large-scale Image Retrieval

arXiv:2604.13268v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong cross-modal reasoning capabilities, yet their potential for vision-only tasks remain

researcharxiv-cs-cl

PersonaVLM: Long-Term Personalized Multimodal LLMs

arXiv:2604.13074v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) serve as daily assistants for millions. However, their ability to generate responses aligned with individual pr

AGMA: Adaptive Gaussian Mixture Anchors for Prior-Guided Multimodal Human Trajectory Forecasting

arXiv:2602.04204v2 Announce Type: replace Abstract: Human trajectory forecasting requires capturing the multimodal nature of pedestrian behavior. However, existing approaches suffer from prior misalig

researcharxiv-cs-cv

15 Apr 2026

MultiDocFusion: Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documents

arXiv:2604.12352v1 Announce Type: new Abstract: RAG-based QA has emerged as a powerful method for processing long industrial documents. However, conventional text chunking approaches often neglect com

15 Apr 2026

SEATrack: Simple, Efficient, and Adaptive Multimodal Tracker

model-releasesarxiv-cs-ai

arXiv:2604.12502v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning (PEFT) in multimodal tracking reveals a concerning trend where recent performance gains are often achieved at the cost

15 Apr 2026

GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Models

arXiv:2601.03416v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have become widely deployed, yet their safety alignment remains fragile under adversarial inputs. Previous

MMRareBench: A Rare-Disease Multimodal and Multi-Image Medical Benchmark

arXiv:2604.10755v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have advanced clinical tasks for common conditions, but their performance on rare diseases remains largely unte

Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models

arXiv:2505.15489v4 Announce Type: replace-cross Abstract: The impact of multimodal misinformation arises not only from factual inaccuracies but also from the misleading narratives that creators delibe

Sense Less, Infer More: Agentic Multimodal Transformers for Edge Medical Intelligence

arXiv:2604.10404v1 Announce Type: cross Abstract: Edge-based multimodal medical monitoring requires models that balance diagnostic accuracy with severe energy constraints. Continuous acquisition of EC

safetyarxiv-cs-lg

AssemLM: Spatial Reasoning Multimodal Large Language Models for Robotic Assembly

model-releasesarxiv-cs-ro

arXiv:2604.08983v1 Announce Type: new Abstract: Spatial reasoning is a fundamental capability for embodied intelligence, especially for fine-grained manipulation tasks such as robotic assembly. While

13 Apr 2026

Visually-Guided Policy Optimization for Multimodal Reasoning

arXiv:2604.09349v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has significantly advanced the reasoning ability of vision-language models (VLMs). However, the

safetyarxiv-cs-ai

13 Apr 2026

A Spatial-Spectral-Frequency Interactive Network for Multimodal Remote Sensing Classification

arXiv:2510.04628v2 Announce Type: replace Abstract: Deep learning-based methods have achieved significant success in remote sensing Earth observation data analysis. Numerous feature fusion techniques

10 Apr 2026

ABMAMBA: Multimodal Large Language Model with Aligned Hierarchical Bidirectional Scan for Efficient Video Captioning

arXiv:2604.08050v1 Announce Type: new Abstract: In this study, we focus on video captioning by fully open multimodal large language models (MLLMs). The comprehension of visual sequences is challenging

researcharxiv-cs-cv

10 Apr 2026

DBMF: A Dual-Branch Multimodal Framework for Out-of-Distribution Detection