Search: “multimodal”

arXiv:2605.30248v1 Announce Type: new Abstract: Image generation models have evolved from text-conditioned pixel synthesis toward multimodal agents endowed with visual comprehension and tool invocatio

agentsarxiv-cs-cv

Tutorials

HD-Prot: A Protein Language Model for Joint Sequence-Structure Modeling with Continuous Structure Tokens

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2512.15133v2 Announce Type: replace-cross Abstract: Proteins inherently possess a consistent sequence-structure duality. The abundance of protein sequence data, which can be readily represented

tutorialsarxiv-cs-ai

29 May 2026

Tutorials

PrismFlow: Residual Dynamics for Flow Matching in Time-Series Generation

arXiv:2605.28867v1 Announce Type: cross Abstract: Generating high-quality time-series data is challenging because real-world signals often exhibit multimodal patterns and multiscale dynamics, includin

tutorialsarxiv-cs-ai

Sample-Efficient Diffusion-based Reinforcement Learning with Critic Guidance

model-releasesarxiv-cs-lg

arXiv:2605.30056v1 Announce Type: cross Abstract: Recent advances in reinforcement learning (RL) have achieved great successes by leveraging the multimodality and exploration capability of diffusion p

Semantic and Visual Evidence for Efficient Long-Video Reasoning: A Solution for the HD-EPIC VQA Challenge

arXiv:2605.29402v1 Announce Type: cross Abstract: Understanding long-form egocentric videos remains challenging for multimodal large language models (MLLMs) due to limited context length and insuffici

SuperVoxelGPT: Adaptive and Ordered 3D Tokenization for Autoregressive Shape Generation

arXiv:2605.29655v1 Announce Type: new Abstract: Autoregressive multimodal large language models (MLLMs) enable 3D generation but struggle to scale to high-resolution shapes due to inadequate 3D tokeni

Assessing Factual Music Comprehension in Large Audio Language Models

arXiv:2511.05550v2 Announce Type: replace-cross Abstract: Large audio language models (LALMs) leverage multimodal representations to generate open-ended answers to natural language queries about audio

Bandwidth-Efficient and Privacy-Preserving Edge-Cloud Many-to-Many Speech Translation

arXiv:2605.28642v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have demonstrated significant potential for speech-to-text translation (S2TT). However, existing deployment par

CuriosAI Submission to the CASTLE Challenge at EgoVis 2026

arXiv:2605.27800v1 Announce Type: new Abstract: CASTLE 2026 asks 185 multiple-choice questions over 600+ hours of synchronized multi-view egocentric video. We explore two approaches on top of a shared

Escape the Language Prior: Mitigating Late-Stage Modality Collapse in Audio Reasoning via Modality-Aware Policy Optimization

arXiv:2605.27741v1 Announce Type: new Abstract: Audio and omni-modal large language models exhibit impressive cross-modal reasoning capabilities. However, applying standard reinforcement learning post

Explaining is Harder Than Predicting Alone: Evaluating Concept-based Explanations of MLLMs as ICL Visual Classifiers

arXiv:2605.28215v1 Announce Type: new Abstract: In-context learning (ICL) enables multimodal large language models (MLLMs) to classify images from a few labelled examples. Yet, how these models use th

Agents

GUI-CIDER: Mid-training GUI Agents via Causal Internalization and Density-aware Exemplar Reselection

arXiv:2605.28534v1 Announce Type: new Abstract: Despite the rapid progress of multimodal large language models in building Graphical User Interface (GUI) agents, their real-world task completion is fu

agentsarxiv-cs-cl

ICG: Improving Cover Image Generation via MLLM-based Prompting and Personalized Preference Alignment

arXiv:2605.27374v1 Announce Type: new Abstract: Recent advances in multimodal large language models (MLLMs) and diffusion models (DMs) have opened new possibilities for AI-generated content. Yet, pers

Mobile-Aptus: Confidence-Driven Proactive and Robust Interaction in MLLM-based Mobile-Using Agents

arXiv:2605.28629v1 Announce Type: new Abstract: Recent advancements in multimodal large language models (MLLMs) have shown exceptional potential in enabling mobile-using agents to autonomously execute

Quantum Machine Learning-based 6G edge Network: Enabling Adaptive Communication and Model Aggregation

arXiv:2605.27417v1 Announce Type: cross Abstract: With the advent of sixth-generation (6G) mobile communication technology, vehicle-to-everything (V2X) communication faces unprecedented challenges in

Rethinking Visual Neglect: Steering via Context-Preference for MLLM Hallucination Mitigation

arXiv:2605.27993v1 Announce Type: new Abstract: Object hallucination remains a primary obstacle to the reliable deployment of Multimodal Large Language Models (MLLMs). Current inference-time mitigatio

researcharxiv-cs-cl

Local Ai

ROVER: Routing Object-Centric Visual Evidence for Grounded Multi-Image Reasoning

arXiv:2605.27959v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have increasingly localized and interleaved visual evidence for deliberative reasoning. Grounding-based appro

local-aiarxiv-cs-ai

Self-Prophetic Decoding to Unlock Visual Search in LVLMs

arXiv:2605.28741v1 Announce Type: new Abstract: Large Vision-Language Models (LVLMs) are rapidly evolving toward true multimodal reasoning, with visual search representing a concrete instantiation of

VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora

arXiv:2605.28683v1 Announce Type: new Abstract: Existing benchmarks have laid the foundation for travel planning agents by establishing API-centric paradigms. However, as the capabilities of Autonomou

VLMs May Not Globally Enhance Human Alignment over LLMs During Natural Reading

arXiv:2605.28818v1 Announce Type: new Abstract: Large language models (LLMs) have become increasingly useful computational models of human language processing, but it remains unclear whether vision-la

CmIVTP: Cross-modal Interaction-based Vessel Trajectory Prediction for Maritime Intelligence

arXiv:2605.26524v1 Announce Type: cross Abstract: Maritime intelligent transportation systems (MITS) are essential for ensuring navigation safety and efficiency in busy waterways. However, accurate ve

safetyarxiv-cs-ai

Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning

arXiv:2605.26292v1 Announce Type: cross Abstract: Parameter-efficient adaptation of vision-language foundation models is crucial for precise multimodal understanding of biomedical images, yet existing

FTibSuite: A Comprehensive Resource Suite for Tibetan Vision-Language Modeling

arXiv:2605.26601v1 Announce Type: new Abstract: Vision-language models have progressed rapidly, but Tibetan remains a severely underserved low-resource language due to the lack of reproducible trainin

safetyarxiv-cs-cv

Furina: Fragmented Uncertainty-Driven Refusal Instability Attack

arXiv:2605.26158v1 Announce Type: cross Abstract: Safety alignment in large language models (LLMs) and multimodal large language models (MLLMs) is commonly assumed to operate as a near-binary threshol

safetyarxiv-cs-ai

IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams

model-releasesarxiv-cs-cv

arXiv:2605.27074v1 Announce Type: new Abstract: Recent multimodal large language models (MLLMs) achieve strong performance on reactive question answering, but real-world streaming assistants require p

Not All Tokens Matter Equally: Dynamic In-context Vector Distillation with Decisive-Token Supervision for Long-form Medical Report Generation

arXiv:2605.27194v1 Announce Type: new Abstract: Distilling demonstration effects into hidden-space interventions offers a lightweight alternative to full finetuning. However, existing multimodal varia

researcharxiv-cs-cl

Real Images, Worse Judgments: Evaluating Vision-Language Models on Concreteness and Imagery

arXiv:2605.27315v1 Announce Type: new Abstract: Visual inputs are often assumed to improve language understanding in multimodal models. We examine this assumption by asking whether vision-language mod

Seeing vs. Believing: Evaluating the Language Bias of Open-Source MLLMs in Counter-Intuitive Scenes

arXiv:2601.07737v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance in mainstream visual understanding tasks, but their ability

Stability Implies Redundancy: Delta Attention Selective Halting for Efficient Long-Context Prefilling

arXiv:2604.18103v2 Announce Type: replace Abstract: Prefilling computational costs pose a significant bottleneck for Large Language Models (LLMs) and Large Multimodal Models (LMMs) in long-context set

TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents

arXiv:2601.05899v2 Announce Type: replace Abstract: Recent breakthroughs in Large Language Models (LLMs) have positioned them as a promising paradigm for agents, with long-term planning and decision-m

VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes

arXiv:2605.26380v1 Announce Type: cross Abstract: Frontier multimodal large language models (MLLMs) have been reported to achieve over 90% accuracy on fine-grained perception benchmarks. However, such

A Comprehensive Dataset for Human vs. AI Generated Image Detection

arXiv:2601.00553v2 Announce Type: replace-cross Abstract: Multimodal generative AI systems like Stable Diffusion, DALL-E, and MidJourney have fundamentally changed how synthetic images are created. Th

Adversarial Orthogonal Disentanglement for LVLM Hallucination Mitigation

arXiv:2605.25377v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have advanced multimodal understanding, yet their reliability is limited by hallucination, where generated conten

AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions

arXiv:2605.25707v1 Announce Type: new Abstract: Autonomous computer use agents that powered by multimodal large language models (MLLMs) are emerging as capable assistants for completing complex digita

AvAtar: Learning to Align via Active Optimal Transport

arXiv:2605.24395v1 Announce Type: new Abstract: Alignment plays a fundamental role in many machine learning problems, such as multi-network analysis, multimodal learning, and point cloud registration.

safetyarxiv-cs-lg

Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory

arXiv:2605.24602v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) frequently suffer from object hallucinations, yet the visual perceptual mechanism underlying this failure rem

Design, Control, and Motion Strategy for DELTA: Transformable Multilink Multirotor for Air-Ground Hybrid Locomotion and Manipulation

arXiv:2403.06636v2 Announce Type: replace Abstract: In recent years, multimodal locomotion capabilities have enabled robots to maneuver in both terrestrial and aerial domains. However, most of these r

researcharxiv-cs-ro

GeoMathCode: Understanding Interleaved Math-Code Reasoning for Geometry Problem Solving

arXiv:2605.25384v1 Announce Type: new Abstract: Mathematical reasoning is a hallmark of human intelligence, requiring logical deduction, symbolic manipulation, and abstract thinking. Recent multimodal

researcharxiv-cs-cl

MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence

arXiv:2505.23764v3 Announce Type: replace-cross Abstract: Spatial intelligence is essential for multimodal large language models (MLLMs) operating in the complex physical world. Existing benchmarks, h

PathMem: Toward Cognition-Aligned Memory Transformation for Pathology MLLMs

arXiv:2603.09943v2 Announce Type: replace Abstract: Computational pathology demands both visual pattern recognition and dynamic integration of structured domain knowledge, including taxonomy, grading

VEN-VL: A Visual Ensemble MoE Framework for Effective and Efficient Multi-Modal Understanding

arXiv:2605.25952v1 Announce Type: cross Abstract: Despite the remarkable progress achieved by recent efficient methods in accelerating multimodal understanding, they still suffer from noticeable perfo

safetyarxiv-cs-ai

Local Ai

CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs

arXiv:2605.23344v1 Announce Type: cross Abstract: Large Vision-Language Models have shown strong multimodal reasoning capabilities, yet they remain susceptible to object hallucinations when language p

local-aiarxiv-cs-ai

GazeBehavior Annotation Toolkit (GBAT): AI-powered toolkit for automatic annotation of egocentric eye-tracking and video data of child-caregiver interaction

arXiv:2605.22962v1 Announce Type: new Abstract: Video recordings of child-caregiver interactions enable investigation of attentional dynamics during naturalistic behavior. Such multimodal recording al

How Far Are We from Generating Missing Modalities with Foundation Models?

arXiv:2506.03530v3 Announce Type: replace-cross Abstract: Multimodal foundation models have demonstrated impressive capabilities across diverse tasks. However, their potential as plug-and-play solutio

Mitigating Object Hallucinations via Sentence-Level Early Intervention

arXiv:2507.12455v3 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have revolutionized cross-modal understanding but continue to struggle with hallucinations - fabricated con

PGT: Procedurally Generated Tasks for improving visual grounding in MLLMs

arXiv:2605.23883v1 Announce Type: cross Abstract: Despite remarkable progress in Multimodal Large Language Models (MLLMs), these models still struggle with fine-grained understanding tasks. In this wo

Agents

RS2AD-LiDAR: End-to-End Autonomous Driving LiDAR Data Generation from Roadside Sensor Observations

arXiv:2605.23406v1 Announce Type: new Abstract: End-to-end autonomous driving solutions, which directly process multimodal sensory data and output fine-grained control commands, have gradually become

agentsarxiv-cs-cv

TCAP: Tri-Component Attention Profiling for Unsupervised Backdoor Detection in MLLM Fine-Tuning

arXiv:2601.21692v2 Announce Type: replace Abstract: Fine-Tuning-as-a-Service (FTaaS) facilitates the customization of Multimodal Large Language Models (MLLMs) but introduces critical backdoor risks vi