Search: “multimodal”

A Fast Screening Approach for High-dimensional Outcomes and High-dimensional Predictors

arXiv:2606.03018v1 Announce Type: cross Abstract: Modeling interactions among multimodal, high-dimensional data is intrinsically challenging due to ultra-high dimensionality and complex dependence str

researcharxiv-cs-lg

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

A unified multi-task framework enables interpretable chest radiograph analysis

DGX agent

arXiv:2606.03417v1 Announce Type: new Abstract: While multimodal deep learning has advanced medical imaging analysis, existing black-box systems extcolor{black}{may remain confined to isolated tasks,

local-aiarxiv-cs-cv

3 Jun 2026

Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model Enhancement

arXiv:2412.01282v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) bring powerful understanding and reasoning capabilities to multimodal tasks. Meanwhile, the great need for capab

Tutorials

Distilling Answer-Set Programming Rules from LLMs for Neurosymbolic Visual Question Answering

arXiv:2606.03269v1 Announce Type: new Abstract: Visual Question Answering (VQA) is the task of answering questions about images, requiring the integration of multimodal input and reasoning. Modular ap

tutorialsarxiv-cs-ai

Local Ai

DMT-CBT: Longitudinal Therapeutic State Modeling for CBT Counseling

arXiv:2606.03132v1 Announce Type: new Abstract: Large language models (LLMs) have shown growing potential for Cognitive Behavioral Therapy (CBT) counseling. However, most existing approaches still for

local-aiarxiv-cs-cl

Investigating Adversarial Robustness of Multi-modal Large Language Models

model-releasesarxiv-cs-cv

arXiv:2606.03713v1 Announce Type: new Abstract: Multi-modal Large Language Models (MLLMs) achieve strong performance on vision-language tasks, but incorporating visual inputs through a vision encoder

MIND: Multi-rationale INtegrated Discriminative Reasoning Framework for Multi-modal Large Models

arXiv:2512.05530v2 Announce Type: replace Abstract: Recently, multimodal large language models (MLLMs) have been widely applied to reasoning tasks. However, they suffer from limited multi-rationale se

QUIVER: Quantum-Informed Views for Enhanced Representations in Large ML Models

arXiv:2606.02785v1 Announce Type: new Abstract: Large machine learning models benefit substantially from multimodal inputs that provide a complementary view of the same example. We introduce QUIVER (Q

See, Infer, Intervene: Proactive World Modeling for Goal-Oriented Social Intelligence

model-releasesarxiv-cs-cl

arXiv:2606.03371v1 Announce Type: new Abstract: Multimodal retail agents should not only recognize what a customer is doing, but also decide whether and how to assist before an explicit request is mad

VistaHop: Benchmarking Multi-hop Visual Reasoning for Visual DeepSearch

arXiv:2606.03273v1 Announce Type: cross Abstract: Visual DeepSearch requires multimodal large reasoning model (MLRM) agents to answer complex visual queries by repeatedly inspecting image regions, gro

World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoning

arXiv:2606.03603v1 Announce Type: cross Abstract: World models and multimodal large language models (MLLMs) provide complementary capabilities for predicting future outcomes from static visual observa

safetyarxiv-cs-cl

AdaCodec: A Predictive Visual Code for Video MLLMs

arXiv:2606.02569v1 Announce Type: cross Abstract: Video is temporally redundant: adjacent frames usually share most objects, background, and layout. Yet existing video multimodal large language models

Beyond Visual Memory: Mechanistic Diagnostics of Latent Visual Reasoning

arXiv:2606.01287v1 Announce Type: cross Abstract: Recent latent visual reasoning methods achieve substantial gains by inserting continuous latent tokens into multimodal language models. These gains ar

Applications

CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluations

arXiv:2606.00123v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have shown strong performance on public medical benchmarks, yet existing evaluations often remain weak proxie

applicationsarxiv-cs-ai

Community-Aware Assessment of Social Textual Engagement and Resonance: A Human-Centric Perspective on User-Generated Content Evaluation

arXiv:2606.01897v1 Announce Type: new Abstract: Traditional Video Quality Assessment (VQA) focuses narrowly on aesthetic fidelity, overlooking the complex social dynamics that define quality in User-G

Decentralized Instruction Tuning: Conflict-Aware Splitting and Weight Merging

arXiv:2606.01717v1 Announce Type: new Abstract: Instruction tuning aligns large language models, including multimodal ones, with diverse user intents, but scaling to heterogeneous mixtures is hindered

ELF: A Family of Encoder-Free ECG-Language Models

arXiv:2601.18798v2 Announce Type: replace-cross Abstract: ECG-Language Models (ELMs) extend recent advances in Multimodal Large Language Models (MLLMs) to automated ECG interpretation. However, most e

Encoded but Not Routed: Explaining the Table-Chart Gap in Scientific Claim Verification

arXiv:2606.01679v1 Announce Type: new Abstract: Multimodal LLMs are increasingly used to assist scientific peer review, where a core requirement is verifying whether claims in a paper are supported by

researcharxiv-cs-cl

ES-Merging: Biological MLLM Merging via Embedding Space Signals

arXiv:2603.14405v2 Announce Type: replace-cross Abstract: Biological multimodal large language models (MLLMs) have emerged as powerful foundation models for scientific discovery. However, existing mod

FastSLM: Hierarchical Temporal Abstraction for Efficient Long-Form Speech Adaptation

arXiv:2601.06199v3 Announce Type: replace-cross Abstract: Scaling Multimodal Large Language Models (MLLMs) to long-form speech is bottlenecked by the explosive growth of input tokens. Unlike images or

FlowNar: Scalable Streaming Narration for Long-Form Videos

arXiv:2606.00620v1 Announce Type: new Abstract: Recent Large Multimodal Models (LMMs), primarily designed for offline settings, are ill-suited for the dynamic requirements of streaming video. While re

Geometric Erasure by Contrastive Velocity Matching in Rectified Flows

arXiv:2606.00140v1 Announce Type: cross Abstract: While the rapid adoption of multimodal generative models offers immense potential, it has also increased the risks of harmful content synthesis, deepf

Hyperbolic and Evidence-Prioritized Experts for Large Vision-Language Models

arXiv:2606.00275v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have demonstrated impressive performance on multimodal tasks through scaled architectures and extensive training.

LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models

arXiv:2606.00573v1 Announce Type: new Abstract: Vision-language models (VLMs) deliver strong multimodal reasoning capabilities, but their large computational cost and high parameter counts make deploy

Med-Scout: Curing MLLMs' Geometric Blindness in Medical Perception via Geometry-Aware RL Post-Training

arXiv:2601.23220v2 Announce Type: replace-cross Abstract: Despite recent Multimodal Large Language Models (MLLMs)' linguistic prowess in medical diagnosis, we find even state-of-the-art MLLMs suffer f

Modeling Robotics Dataset Construction as an Artifact-Based Build Process

arXiv:2606.00162v1 Announce Type: cross Abstract: Robotic systems generate large volumes of multimodal sensor data, but converting ROS bag recordings into machine learning datasets is often handled by

Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events

arXiv:2606.02522v1 Announce Type: cross Abstract: Video multimodal large language models (MLLMs) have made rapid progress on general and long-form video understanding, yet their ability to preserve br

PolarMem: A Training-Free Polarized Latent Graph Memory for Verifiable Vision-Language Models

arXiv:2602.00415v2 Announce Type: replace Abstract: Memory is not merely a storage mechanism for intelligent systems, but a structure for organizing evidence and constraining belief. This is especiall

Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearning

arXiv:2503.11832v5 Announce Type: replace Abstract: Recent vision language models (VLMs) have made remarkable strides in generative modeling with multimodal inputs, particularly text and images. Howev

Silent Failures in Physical AI: A Literature Review of Runtime Action Authorization for Autonomous Systems

arXiv:2606.00090v1 Announce Type: cross Abstract: Physical AI systems increasingly map multimodal observations, language instructions, and learned world representations into physically consequential a

StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning

arXiv:2606.00148v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) often know the rule but pick the wrong answer: on abstract visual reasoning (AVR) tasks, a model can describe

The Right Inference Strategy Is All You Need: Nearly Training-Free Domain-Wise Inference for EgoCross Challenge

arXiv:2606.00829v1 Announce Type: new Abstract: EgoCross evaluates multimodal large language models on egocentric video question answering under substantial domain shift, where test videos come from s

Tutorials

Video Reasoning without Training

arXiv:2510.17045v2 Announce Type: replace-cross Abstract: Video reasoning using Large Multimodal Models (LMMs) relies on costly reinforcement learning (RL) and verbose chain-of-thought, resulting in s

tutorialsarxiv-cs-ai

When Meaning Travels: A Granular Lens on Hybrid-MoE's Role in Idiomatic Understanding for Language Models

arXiv:2606.01671v1 Announce Type: new Abstract: In the contemporary epoch of multilingual education, learning idioms provides a fascinating gateway towards creativity, cultural values, historical cont

safetyarxiv-cs-cl

World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications

arXiv:2606.00133v1 Announce Type: new Abstract: World models, internal simulators that learn the structure and dynamics of an environment, have emerged as a central paradigm in the pursuit of artifici

safetyarxiv-cs-lg

D-SECURE: Dual-Source Evidence Combination for Unified Reasoning in Misinformation Detection

arXiv:2602.14441v2 Announce Type: replace Abstract: Multimodal misinformation increasingly mixes realistic im-age edits with fluent but misleading text, producing persuasive posts that are difficult t

Does Visual Information Play a Decisive Role in Vision-Language-Action Model Driving Behavior?

arXiv:2605.31041v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have demonstrated promising capability in autonomous driving, highlighting the potential of unified multimodal arc

Local Ai

iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement Learning

arXiv:2605.31096v1 Announce Type: new Abstract: While visually grounded Chain-of-Thought (CoT) has emerged as a promising paradigm to enhance fine-grained perception in multimodal large language model

local-aiarxiv-cs-cv

Agents

Learning to Adapt: Self-Improving Web Agent via Cognitive-Aware Exploration

arXiv:2605.31365v1 Announce Type: new Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have led to promising progress in web agents. However, existing web agents often rely on han

agentsarxiv-cs-ai

MineExplorer: Evaluating Open-World Exploration of MLLM Agents in Minecraft

model-releasesarxiv-cs-cl

arXiv:2605.30931v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have shown strong capabilities in perception, reasoning, and action generation. However, their ability to susta

The Regularizing Power of Language-Training Deepfake Detectors

model-releasesarxiv-cs-cv

arXiv:2605.31192v1 Announce Type: new Abstract: Recently, thanks to the advent of Multimodal-LLMs, deepfake detectors are striving not only to be generalizable but also interpretable. We propose that

Local Ai

Vanilla ViT for Automotive Point Cloud Semantic Segmentation

arXiv:2605.31177v1 Announce Type: new Abstract: Plain Transformers have become the de-facto architecture for processing text, audio, image, and video, offering a unified backbone for multimodal learni

local-aiarxiv-cs-cv

Wall-OSS-0.5 Technical Report

arXiv:2605.30877v1 Announce Type: new Abstract: Large-scale Vision-Language-Action (VLA) pretraining is increasingly adopted as the foundation for robot policies, yet the evidence for pretrained VLAs

safetyarxiv-cs-ro

4DPC^2hat: Towards Dynamic Point Cloud Understanding with Failure-Aware Bootstrapping

arXiv:2602.03890v2 Announce Type: replace Abstract: Point clouds provide a compact and expressive representation of 3D objects, and have recently been integrated into multimodal large language models

AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling

arXiv:2605.29488v1 Announce Type: cross Abstract: Conditional human motion generation remains a fundamental challenge in computer vision and robotics. Despite significant progress, current methods are

CogniVerse: Revolutionizing Multi-Modal Retrieval-Augmented Generation with Cognitive Reflection and Geometric Reasoning

arXiv:2605.29602v1 Announce Type: new Abstract: Multi-modal Retrieval-Augmented Generation (MMRAG) has emerged as a powerful paradigm for enhancing Multimodal Large Language Models in knowledge-intens

CrystalXRD-Bench: Benchmarking Vision-Language Models for XRD Peak Indexing Across Diverse Crystalline Materials

arXiv:2605.29446v1 Announce Type: new Abstract: Miller-index identification from powder XRD patterns requires capabilities untested by existing multimodal benchmarks: the model must read a narrow peak

Diffusion-based learning framework for Constrained Nonconvex Optimization with Weighted Bootstrapped Refinement

arXiv:2502.10330v4 Announce Type: replace Abstract: Recent advances in diffusion models show promising potential to accelerate nonconvex problem solving by leveraging their multimodality. However, mos