Search: “multimodal”

DIPrune: Task-Aware Token Pruning with Dual Importance for Efficient Multimodal Language Models

arXiv:2610.08341v1 Announce Type: new Abstract: Recent training-free pruning approaches for Multimodal Large Language Models (MLLMs) effectively cut computational overhead by exploiting visual redunda

7 Oct 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Nonlocal Hamiltonian Dynamics on Sparse Levy Graphs: Spectral Analysis and Multimodal Sampling

DGX agent

arXiv:2610.06904v1 Announce Type: cross Abstract: We develop a sparse graph method for transporting probability mass toward multimodal target distributions through damped nonlocal Hamiltonian dynamics

researcharxiv-cs-lg

7 Oct 2026

Anatomy-aware Fine-grained Multimodal Fusion for Laryngopharyngeal Cancer T-Staging Prediction Using CT and Radiology Report

arXiv:2610.06837v1 Announce Type: new Abstract: Accurate T-staging is crucial for guiding personalized treatment strategies for laryngopharyngeal cancer. However, current clinical practice relies on i

safetyarxiv-cs-cv

Conditional Trajectory Peaks: Single-Pass Multimodal Policies over Action Chunks

arXiv:2610.06104v1 Announce Type: new Abstract: Multimodal imitation learning requires diverse executable futures under the same observation and consistent behavior across replanning cycles. We presen

safetyarxiv-cs-ro

DREAM: Dynamic Resolution Assignment For Multimodal Multi-agent Debate

arXiv:2610.05615v1 Announce Type: cross Abstract: Multi-agent debate (MAD) has emerged as an effective paradigm to improve the reasoning capabilities of large language models (LLMs) and is increasingl

agentsarxiv-cs-cv

fMRI-TAMCL: Text-Anchored Supervised Multimodal Contrastive Learning for fMRI-Based Brain Disorder Classification

arXiv:2610.05880v1 Announce Type: new Abstract: Resting-state fMRI is important in the classification of brain disorders, but highly multimodal and exhibits strong multisite heterogeneity. Existing me

Recursive Self-Improvement in Unified Multimodal Models

arXiv:2610.03002v1 Announce Type: new Abstract: Unified multimodal models (UMMs) understand and generate both text and images, which lets a model produce its own training data. Existing self-improveme

researcharxiv-cs-cl

5 Oct 2026

FairSSL: Fair Multimodal Self-Supervised Learning

arXiv:2508.16748v2 Announce Type: replace-cross Abstract: Prevalent multimodal self-supervised learning (SSL) methods rely on the redundancy assumption: that different views share substantial task-rel

safetyarxiv-cs-ai

Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation

arXiv:2610.02123v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures scale model capacity through sparse computation, routing each token through only a small subset of experts. In th

Applications

ScaffoldM3C: A Multimodal Sequential Monte Carlo Framework for Generative Stable Construction Planning

arXiv:2610.00487v1 Announce Type: cross Abstract: Autonomously constructing physically realizable 3D structures remains a significant challenge due to combinatorial action spaces, interchangeable comp

applicationsarxiv-cs-lg

Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents

arXiv:2610.01892v1 Announce Type: cross Abstract: Multimodal agents commonly generate free-form reasoning before each action. For small models, limited model capacity can result in lengthy reasoning t

researcharxiv-cs-ai

From Charts to Code: A Hierarchical Benchmark for Multimodal Models

arXiv:2510.17932v5 Announce Type: replace-cross Abstract: We introduce Chart2Code, a new benchmark for evaluating the chart understanding and code generation capabilities of large multimodal models (L

1 Oct 2026

Ranking-Aware Prompt Optimization for Multimodal Clinical Diagnosis

arXiv:2609.40361v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) are rapidly advancing clinical diagnosis, yet their adaptation pipelines remain anchored to accuracy-based ob

researcharxiv-cs-cl

1 Oct 2026

ShieldCLIP: Selective Safety Alignment for Harmful Content Mitigation in Multimodal Foundation Models

arXiv:2609.39688v1 Announce Type: cross Abstract: Multimodal encoders such as CLIP underlie many downstream systems, but their web-scale training data embed harmful associations that safety alignment

safetyarxiv-cs-ai

1 Oct 2026

Calibrate the Decisions That Change the Future: On-Policy Post-Training Quantization for Multimodal Large Language Models

arXiv:2609.36828v1 Announce Type: new Abstract: Post-training quantization (PTQ) lowers deployment cost for multimodal large language models, but calibration typically reconstructs fixed sequences wit

30 Sept 2026

MMSkillRisk: Can Agents Stay Safe When Multimodal Skills Become Traps?

arXiv:2609.35912v1 Announce Type: cross Abstract: Agent skills are shareable packages of procedural instructions, tools, and examples. Multimodal skills additionally include visual references that age

30 Sept 2026

Binding Multiple Modalities via Multimodal Wasserstein Barycenter

model-releasesarxiv-cs-lg

arXiv:2609.33800v1 Announce Type: new Abstract: Multimodal learning beyond two modalities commonly leverages a specific modality (e.g., text) to bind other modalities. However, how to establish a more

EngIntervene: Benchmarking Multimodal Engineering State Understanding and Design Intervention Reasoning

arXiv:2609.33261v1 Announce Type: new Abstract: Multimodal engineering benchmarks largely evaluate static understanding, such as recognizing components, interpreting diagrams, or answering technical q

Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding

arXiv:2609.34672v1 Announce Type: new Abstract: Fine-grained visual understanding requires models to recognize small details within complex images. Multimodal on-policy self-distillation (OPSD) addres

safetyarxiv-cs-cv

Learning Multimodal Embeddings with Evidence-Aligned Readout

arXiv:2609.33659v1 Announce Type: new Abstract: Multimodal large language models can expose task-relevant evidence through generation, but producing useful evidence does not by itself determine how it

OmniSmartHome: A Multimodal Reasoning Benchmark for Smart-Home Agents

arXiv:2609.32569v1 Announce Type: new Abstract: Smart-home assistants are expected to handle diverse, realistic requests that arise in daily life. In such interactions, users often rely on the surroun

ORAV: Benchmarking Audio-Video Generation from Multimodal Contexts

arXiv:2609.34843v1 Announce Type: new Abstract: Audio-video generation using heterogeneous multimodal references has emerged as a new challenge, requiring both compositional control over generation an

PACGNet: Pyramidal Adaptive Cross-Gating Network for Multimodal Object Detection in Aerial Imagery

arXiv:2512.18291v3 Announce Type: replace Abstract: Object detection in aerial imagery is challenging because targets are often small, backgrounds are cluttered, and imaging conditions vary substantia

Retrieved but Not Delivered: Multimodal Memory Delivery for Long-Term Agents

arXiv:2609.32590v1 Announce Type: new Abstract: Work on memory for multimodal agents optimizes what is written, updated and retrieved. Between retrieval and the answer, however, is a stage that multim

agentsarxiv-cs-cv

SafeMol: Dual-Modality Safety Alignment for Molecular Multimodal Models

model-releasesarxiv-cs-lg

arXiv:2609.33640v1 Announce Type: new Abstract: Molecular multimodal models support diverse understanding and generation tasks but may introduce safety vulnerabilities when handling hazardous molecule

Source-Lifted Flow Matching for Intervenable Multimodal Imitation

model-releasesarxiv-cs-ro

arXiv:2607.10206v2 Announce Type: replace Abstract: Flow-matching policies are promising for imitation learning because they model complex multimodal action distributions. However, their stochasticity

The Plot Twist: Jailbreaking Unified Multimodal Models with a Three-Act NarrativeAttack

arXiv:2509.26473v2 Announce Type: replace Abstract: Unified Multimodal Understanding and Generation Models (UMMs) increasingly combine visual understanding and image generation within a single interac

28 Sept 2026

ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs

arXiv:2609.31448v1 Announce Type: cross Abstract: Clinical prediction models estimate risk from patient measurements, while large language models support medical text understanding and question answer

28 Sept 2026

AgenticCADedit: A Stateful, Tool-Mediated Agentic Approach to Multimodal 3D CAD Editing

arXiv:2609.29621v1 Announce Type: new Abstract: Computer-aided design is central to industrial manufacturing, and much of a designer's daily work consists of editing existing models from multimodal re

25 Sept 2026

ICE: Task-Aligned Clifford Latent Fields for Multimodal Graph Foundation Models

arXiv:2609.29398v1 Announce Type: new Abstract: Multimodal attributed graphs connect entities, visual content, language, and observed relations. Learning one foundation across such graphs requires mor

researcharxiv-cs-lg

25 Sept 2026

UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm

arXiv:2609.27915v1 Announce Type: new Abstract: Despite remarkable advancements in multimodal large language models (MLLMs), their fine-grained visual understanding is constrained by a primary relianc

24 Sept 2026

VCMM: Variance-Calibrated Momentum for Multimodal Learning

arXiv:2609.27577v1 Announce Type: new Abstract: Multimodal joint training often suffers from modality imbalance, where a dominant modality suppresses the optimization of others. Existing methods mainl

safetyarxiv-cs-lg

24 Sept 2026

Local Ai

BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception

arXiv:2609.25891v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) often struggle with fine-grained visual perception when processing complete images, as critical evidence may on

local-aiarxiv-cs-cv

From Research Frontier to Laboratory Bench: Design of a Four-Tier Experimental Teaching System for Multimodal Medical Image Intelligent Diagnosis

arXiv:2609.22790v1 Announce Type: new Abstract: Undergraduate programmes in intelligent medical engineering are expanding, yet laboratory curricula lag behind the multimodal, long-tailed, and distribu

safetyarxiv-cs-ai

ICDAR2026 Competition on Multimodal Reasoning over Documents in Multiple Domains

arXiv:2609.25055v1 Announce Type: new Abstract: In this report we present results of the ICDAR2026 Competition on Multimodal Reasoning over Documents in Multiple Domains. This competition aimed to adv

agentsarxiv-cs-cl

Learning to Plan in Human-Robot Collaboration: Multimodal Reinforcement Learning for Adaptive Interaction

arXiv:2609.25274v1 Announce Type: new Abstract: Robot assistants for older adults and people with disabilities need to perform collaborative tasks with users effectively. The core component of these s

safetyarxiv-cs-ro

MatchFusion: Explicit-Implicit Instance Matching for Spatio-Temporal Multimodal Autonomous Driving

arXiv:2609.25860v1 Announce Type: cross Abstract: Sparse instance representations provide a compact interface for spatial LiDAR-camera and temporal past-current interaction in multimodal perception an

safetyarxiv-cs-ro

Virtual Encoders in Multimodal Transformers

arXiv:2609.26513v1 Announce Type: new Abstract: Multimodal language models traditionally rely on dedicated perceptual encoders to construct task-usable representations. More integrated architectures h

Dynamic Thermal Gaussians: Multimodal 4D Gaussian Splatting

arXiv:2609.24531v1 Announce Type: new Abstract: Thermography plays a vital role in military and broader thermal analysis applications. Recent progress in 3D thermal reconstruction has extended tempera

GeoBalance: Geometry-Aware Monitoring and Reconstruction with Asymmetric Optimization for Balanced Multimodal Learning

arXiv:2609.23533v1 Announce Type: new Abstract: Multimodal classifiers can converge to modality-dominant solutions in which one modality dominates the joint prediction, suppressing the learning of oth

Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models

arXiv:2609.23715v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) incur substantial computational overhead due to the reliance on hundreds of visual tokens to represent images.

MIGU: Multimodal Instruction Grounding under Uncertainty for Manipulation Planning

model-releasesarxiv-cs-ro

arXiv:2609.24995v1 Announce Type: new Abstract: Understanding natural human instructions is crucial for deploying robots in human-centric environments. We study multimodal instruction grounding, where

MM-ContextFold: Context Folding for Multimodal Agentic Retrieval

arXiv:2609.23121v1 Announce Type: new Abstract: Multimodal Agentic Retrieval (MAR) requires agents to solve complex information-seeking tasks by iteratively invoking external tools. Typical frameworks

agentsarxiv-cs-cv

Summarize, Judge, Refine: Decoupled Content Understanding and Policy Learning for Multimodal Content Moderation

arXiv:2609.22094v1 Announce Type: new Abstract: Content moderation systems traditionally entangle multimodal understanding with policy-specific classification, requiring full pipeline retraining for e

safetyarxiv-cs-cl

DRT: Dense Reasoning Trace for Efficient and Grounded Multimodal Reasoning

arXiv:2609.21675v1 Announce Type: new Abstract: Despite the remarkable progress in Multimodal Large Language Models (MLLMs), prevailing Chain-of-Thought (CoT) paradigms remain confined to the natural-

21 Sept 2026

M2G-LLM: Enhancing Clinical Prediction via Multimodal Graph Reasoning and LLM Context Injection

arXiv:2609.21164v1 Announce Type: new Abstract: Integrating diverse data modalities --- such as clinical notes, laboratory results, and medical imaging --- is essential for advancing clinical decision

safetyarxiv-cs-lg

21 Sept 2026

PolyBridgeBench: Benchmarking Multimodal LLMs for Physics-Grounded Bridge Design

arXiv:2609.21493v1 Announce Type: new Abstract: Multimodal large language models, or MLLMs, perform well at visual understanding and structured generation, yet these capabilities do not establish whet

21 Sept 2026

Applications

Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts