Search: “multimodal”

When Irrelevant Text Matters: Affine Margin Shifts in Multimodal Large Language Models

arXiv:2608.19208v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are frequently exposed to auxiliary textual context, the impact of which on visually grounded tasks remains und

safetyarxiv-cs-cl

21 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models

DGX agent

arXiv:2608.17895v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) have made significant strides in visual comprehension, their ability to reason about text-dense, profes

model-releasesarxiv-cs-ai

19 Aug 2026

FuseLIP: Multimodal Embeddings via Early Fusion of Discrete Tokens

arXiv:2506.03096v2 Announce Type: replace Abstract: Contrastive language-image pre-training aligns features of text-image pairs in a common latent space via distinct encoders for each modality. While

18 Aug 2026

Multi-Granularity Sentiment Integration for LLM-Based Multimodal Sentiment Analysis

arXiv:2608.16201v1 Announce Type: new Abstract: Multimodal sentiment analysis (MSA) aims to predict sentiment polarity and intensity from heterogeneous inputs such as text, audio, and vision. While la

local-aiarxiv-cs-lg

18 Aug 2026

Removing Temporal Note Redundancy Improves Multimodal Reinforcement Learning for Medicine

arXiv:2608.14157v1 Announce Type: new Abstract: Mechanical ventilation is a critical life-support intervention, requiring dynamic adjustments to ventilator settings as a patient's condition evolves. W

local-aiarxiv-cs-ai

17 Aug 2026

SPARGen: Unifying Spatial Perception and Reasoning through Native Multimodal Generation

arXiv:2608.14138v1 Announce Type: cross Abstract: Spatial perception and reasoning from visual observations require recovering geometric structure, establishing correspondences, and understanding spat

17 Aug 2026

Adjustable Text-Guided Backdoor Attacks with Natural-Word Triggers on Multimodal Pretrained Models

arXiv:2604.05809v2 Announce Type: replace-cross Abstract: This paper presents Text-Guided Backdoor (TGB), an adjustable backdoor attack against multimodal pretrained models that uses natural-word trig

applicationsarxiv-cs-lg

14 Aug 2026

LEMUR: Latent Entropy-aware Multimodal Unlearning via Visual-anchored Reasoning Redirection

arXiv:2608.11691v1 Announce Type: cross Abstract: Reinforcement-learning (RL) post-training equips multimodal large reasoning models (MLRMs) with exploratory chains of thought (CoT), substantially imp

researcharxiv-cs-cl

13 Aug 2026

VLM2Rec: Resolving Modality Collapse in Vision-Language Model Embedders for Multimodal Sequential Recommendation

arXiv:2603.17450v2 Announce Type: replace-cross Abstract: Sequential Recommendation (SR) in multimodal settings typically relies on small frozen pretrained encoders, which limits semantic capacity and

13 Aug 2026

MultiModal Code-Switching: Interleaving Visual Objects into Language for Explicit Object-Level Alignment

arXiv:2608.11167v1 Announce Type: cross Abstract: Existing Multimodal Large Language Models (MLLMs) predominantly rely on image-text pairs for modality alignment pretraining, mapping global image repr

local-aiarxiv-cs-cl

12 Aug 2026

VERDICT: Training-Free Step-Wise Verification of Multimodal Reasoning via Disagreement-Aware Consensus

arXiv:2608.10665v1 Announce Type: new Abstract: Multimodal large language models often generate reasoning chains containing subtle errors that lead to incorrect answers. Current verification approache

12 Aug 2026

Discovering Diverse Planning Policies for Multimodal Embodied Agents with Quality-Diversity Optimization

arXiv:2608.08523v1 Announce Type: new Abstract: Multimodal embodied agents are increasingly required to solve long-horizon tasks by integrating visual observations, textual goals, and interaction hist

Multi-Branch Policy Optimization for Multimodal Large Language Models

arXiv:2608.07581v1 Announce Type: cross Abstract: Group-based reinforcement learning methods for multimodal large language models typically rely on trajectory-level credit assignment that applies a si

safetyarxiv-cs-ai

Multimodal Federated Learning under Dual-Axis Modality Missingness

arXiv:2608.09240v1 Announce Type: cross Abstract: Multimodal federated learning (FL) supports collaborative modeling in privacy-sensitive health-sensing and medical settings, but realistic deployments

local-aiarxiv-cs-ai

StructReward: Efficient Structured Process Rewards for Self-Correcting Multimodal Reasoning

arXiv:2608.08326v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as an effective approach for improving multimodal reasoning. However, most existing me

safetyarxiv-cs-ai

UniSpace: Unified Visual Representation and Scalable Multimodal Modeling

arXiv:2608.08676v1 Announce Type: cross Abstract: Semantic vision encoders have become a central visual interface for multimodal understanding and semantic conditioning in image generation. However, t

CertBind from Multimodal Connectivity to Certifiable Retrieval Decisions

arXiv:2608.06516v1 Announce Type: cross Abstract: Lightweight connectors make frozen multimodal encoders composable at the representation level. Deployment exposes a second problem at the level of tas

applicationsarxiv-cs-ai

10 Aug 2026

Surg-UniWorld: A Unified Surgical World Model with Multimodal Control Experts

arXiv:2608.06770v1 Announce Type: new Abstract: Controllable surgical world models can provide a generative foundation for surgical artificial intelligence and simulation by synthesizing realistic ins

10 Aug 2026

Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval

arXiv:2608.06060v1 Announce Type: new Abstract: Unified multimodal retrieval aims to identify candidates that satisfy complex user intent expressed through heterogeneous inputs. Although Large Vision-

7 Aug 2026

Seeing Is Not Deciding: Can Multimodal LLMs Act as Effective CEOs?

arXiv:2608.05864v1 Announce Type: new Abstract: Large language models are increasingly applied as autonomous decision-making agents. However, in executive business decisions, existing benchmarks are l

7 Aug 2026

iStructTab: Structured Feature Sequencing for Multimodal Learning of Image and Tabular Data

arXiv:2608.04348v1 Announce Type: cross Abstract: Multimodal learning of images and tabular data is often impaired by ineffective representations, resulting in redundancy, dispersion, and generalizati

6 Aug 2026

RESPClinBench: Benchmarking Multimodal Clinical Decision-Making and Longitudinal Disease Management in Respiratory Specialty Care

model-releasesarxiv-cs-cl

arXiv:2608.04514v1 Announce Type: new Abstract: Background: Respiratory specialty care requires multimodal interpretation, longitudinal risk assessment, guideline-concordant intervention, and whole-co

6 Aug 2026

Adaptive Modality Reliability Diagnosis and Restoration for Robust Multimodal Intent Recognition

arXiv:2608.03475v1 Announce Type: cross Abstract: Multimodal intent recognition combines linguistic, acoustic, and visual evidence, but individual modalities may be noisy, missing, semantically confli

5 Aug 2026

From Generator to Embedder: Harnessing Innate Abilities of Multimodal LLMs via Building Zero-Shot Discriminative Embedding Model

arXiv:2508.00955v3 Announce Type: replace-cross Abstract: Adapting generative Multimodal Large Language Models (MLLMs) into universal embedding models typically demands resource-intensive contrastive

5 Aug 2026

PatTree: a novel approach for automated creation of multimodal, graph-based patient representations for medical classification tasks

arXiv:2608.02692v1 Announce Type: new Abstract: Access to holistic, multimodal data improves the performance of Artificial Intelligence (AI) in medical classification tasks compared to utilizing singl

applicationsarxiv-cs-lg

5 Aug 2026

Cross-Branch Conflict as a Shield: Safeguarding Facial Identities in Unified Multimodal Image Editing

arXiv:2607.16898v2 Announce Type: replace-cross Abstract: Unified multimodal models (UMMs) have recently demonstrated powerful instruction-based image editing capabilities, while also raising serious

safetyarxiv-cs-cl

4 Aug 2026

From Direction to Magnitude: How Multimodal Instruction-Tuning Reorganizes the Geometric Encoding of Identity-Specifying Prompts in Transformer Hidden States

model-releasesarxiv-cs-cl

arXiv:2607.09842v2 Announce Type: replace-cross Abstract: We investigate whether identity-specifying system prompts produce statistically distinguishable geometric fingerprints in the hidden-state tra

4 Aug 2026

Towards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneity

arXiv:2608.00623v1 Announce Type: new Abstract: Multimodal-attributed graphs (MAGs), where nodes carry heterogeneous semantic content across multiple modalities while edges encode relational dependenc

researcharxiv-cs-lg

4 Aug 2026

Beyond Retrieval: Analytic Memory for Multimodal Agents

arXiv:2607.29440v1 Announce Type: new Abstract: Long-term multimodal memory must support not only retrieving relevant information but also computing over observations accumulated across interactions.

3 Aug 2026

Tutorials

PaletteID: Prototype-Composed Semantic Identifiers for Multimodal CTR Prediction

arXiv:2607.29000v1 Announce Type: cross Abstract: Multimodal information can improve the accuracy of click-through rate (CTR) prediction and effectively alleviate item cold-start and long-tail problem

tutorialsarxiv-cs-lg

3 Aug 2026

Understanding Alignment in Multimodal LLMs: A Comprehensive Study

Preference alignment has become a crucial component in enhancing the performance of Large Language Models (LLMs), yet its impact in Multimodal Large Language Models (MLLMs) remains comparatively under

safetyapple-ml-research

3 Aug 2026

Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Models

arXiv:2607.28341v1 Announce Type: new Abstract: While visual token pruning is essential for efficient Multimodal Large Language Models (MLLMs), existing training-free methods suffer from a critical li

local-aiarxiv-cs-cv

Tutorials

FedOGL: Combating Catastrophic Forgetting in Federated Open-World Multimodal Graph Learning

arXiv:2607.27665v1 Announce Type: new Abstract: Federated graph learning enables collaborative training over decentralized graph data without sharing raw graph information. As such risks evolve, clien

tutorialsarxiv-cs-lg

MIND: Multimodal Intent-Driven Network via Diffusion Transformers for Medical Image Fusion

arXiv:2607.28565v1 Announce Type: new Abstract: Medical image fusion aims to integrate complementary information from diverse imaging modalities to support clinical diagnosis. Existing methods typical

safetyarxiv-cs-cv

Theia: Large-Scale Multimodal Captioning and Automated Validation of the Incidents1M Dataset for Data-Free Distillation

arXiv:2607.28269v1 Announce Type: new Abstract: The deployment of Vision-Language Models (VLMs) in critical domains like disaster management requires high-quality multimodal datasets, especially for t

Uncertainty-Aware Multimodal Fusion for Oral Lesion Classification

arXiv:2511.12268v3 Announce Type: replace-cross Abstract: Early detection of oral cancer and potentially malignant diseases is a major challenge in low-resource settings due to the scarcity of annotat

applicationsarxiv-cs-cv

Agents

End-to-End 3-D Spatiotemporal Perception with Multimodal Fusion and V2X Collaboration

arXiv:2512.21831v2 Announce Type: replace Abstract: Multiview cooperative perception and multimodal fusion are essential for reliable 3-D spatiotemporal understanding in autonomous driving, especially

agentsarxiv-cs-cv

29 Jul 2026

From Cellular Responses to Pharmacological Domains: Multimodal Zero-Shot Drug Representation Learning

arXiv:2607.25322v1 Announce Type: new Abstract: Multimodal drug discovery enables drug representation learning beyond chemical structure by incorporating cellular responses such as gene expression and

29 Jul 2026

DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models

arXiv:2607.23944v1 Announce Type: new Abstract: Human visual reasoning typically follows a coarse-to-fine attention process, starting from global scene understanding and gradually focusing on question

safetyarxiv-cs-ai

FedTaste: Topology-Aware Structural Transfer for Multimodal Federated Learning with Missing Modalities

model-releasesarxiv-cs-lg

arXiv:2607.23245v1 Announce Type: cross Abstract: Multimodal Federated Learning is often challenged by arbitrary modality missingness and Non-IID data distributions, which lead to severe representatio

LEGO Co-builder: Exploring Fine-Grained Vision-Language Modeling for Multimodal LEGO Assembly Assistants

arXiv:2507.05515v4 Announce Type: replace Abstract: Vision-language models (VLMs) are facing the challenges of understanding and following multimodal assembly instructions, particularly when fine-grai

Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding

model-releasesarxiv-cs-cv

arXiv:2607.24407v1 Announce Type: new Abstract: Multimodal Large Language Models have made great progress in grounding tasks, yet existing methods still struggle to unify precise localization and comp

MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models

arXiv:2607.22586v1 Announce Type: new Abstract: Key-Value (KV) caching is essential for efficient inference in multimodal large language models (MLLMs), yet its memory footprint grows linearly with co

MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents

arXiv:2607.23870v1 Announce Type: cross Abstract: Smart-city airspace is transforming Uncrewed Aerial Vehicles (UAVs) from passive sensing platforms into cyber-physical decision makers that must follo

Multimodal Surface EMG Hand Gesture Recognition Using Query-Based Transformers for Prosthetic Control

arXiv:2607.22779v1 Announce Type: cross Abstract: Hand gesture recognition via surface electromyography (sEMG) is fundamental to prosthetic control. In this field, deep learning approaches have become

local-aiarxiv-cs-ai

Verification-Notebook Learning for Source-Aware Multimodal Misinformation Detection

arXiv:2607.23581v1 Announce Type: new Abstract: Multimodal misinformation verification is challenging because misleading signals may come from different parts of a post and require different forms of

Tutorials

Look Less, Think Faster: Joint Token-Compute Adaptation for Multimodal LLMs

arXiv:2607.20357v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have recently demonstrated strong performance across vision-language tasks. However, their high inference cost,

tutorialsarxiv-cs-cv

23 Jul 2026

Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?

arXiv:2607.20284v1 Announce Type: new Abstract: The rapid development of multimodal large language models (MLLMs) has introduced a flexible paradigm for remote sensing image scene understanding (RSISU