Search: “multimodal”

arXiv:2605.28115v1 Announce Type: new Abstract: Vision-Language Models (VLMs) face severe memory and latency bottlenecks due to high-resolution visual tokens. While current token reduction methods the

local-aiarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

CogPortrait: Fine-Grained Eye-Region Control in Portrait Animation via Hierarchical Agent Planning

DGX agent

arXiv:2605.28056v1 Announce Type: new Abstract: Portrait animation methods have achieved substantial visual quality and lip synchronization, but fine-grained manipulation of the eye region still faces

model-releasesarxiv-cs-cv

28 May 2026

CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification

arXiv:2508.21046v3 Announce Type: replace Abstract: Recent Vision-Language-Action (VLA) models built on pre-trained Vision-Language Models (VLMs) require extensive post-training, resulting in high com

CPPO: Contrastive Perception Policy Optimization for VLM Agents

arXiv:2601.00501v2 Announce Type: replace Abstract: We introduce CPPO, a Contrastive Perception Policy Optimization method for finetuning vision--language models (VLMs). Reliable perception is a core

safetyarxiv-cs-cv

Cross-Modal Action Recognition in Egocentric Video Using Mamba: Integrating RGB and Hand Skeleton Streams via CLS Token Fusion Strategies

arXiv:2605.24302v2 Announce Type: replace Abstract: Egocentric action recognition is a challenging task due to erratic camera motion, frequent hand occlusion, and the difficulty of maintaining consist

researcharxiv-cs-cv

Cyberbullying Governance on Social Media: A Unified Framework from Content Identification to Intervention

arXiv:2605.27584v1 Announce Type: new Abstract: The proliferation of social media platforms and online communities has inadvertently catalyzed the spread of cyberbullying, hate speech, and other forms

safetyarxiv-cs-ai

Do LLMs Build World Models From Text? A Multilingual Diagnostic of Spatial Reasoning

arXiv:2605.28277v1 Announce Type: new Abstract: Whether large language models (LLMs) construct internal spatial world models from pure-text descriptions remains contested, and whether such capabilitie

ForestHG-Trace: Traceable Long-Horizon Ecological Reasoning over Large-Scale Forest Scenes

arXiv:2605.27590v1 Announce Type: new Abstract: Remote sensing question answering (RS-QA) often requires more than direct semantic prediction, especially in large-scale forest scenes where ecological

From Pixels to Words -- Towards Native One-Vision Models at Scale

arXiv:2605.28820v1 Announce Type: new Abstract: Current vision-language models (VLMs) typically stitch together separate image encoders and language decoders via multi-stage alignment, a modular frame

safetyarxiv-cs-cv

Hierarchical Synthetic Tabular Data Generation: A Hybrid Top-Down and Bottom-Up Framework

arXiv:2605.28198v1 Announce Type: new Abstract: Existing approaches for synthetic tabular data generation are based on either purely generative models or LLMs, both of which struggle with data heterog

safetyarxiv-cs-lg

Latent-Conditioned Parameterized Quantum Circuits as Universal Approximators for Distributions over Quantum States

arXiv:2605.28690v1 Announce Type: cross Abstract: Many applications in quantum simulation, quantum chemistry, and quantum machine learning require not a single quantum state but an ensemble of states

researcharxiv-cs-lg

MACReD: A Multi-Agent Collaborative Reasoning Framework for Reaction Diagram Parsing

arXiv:2605.28077v1 Announce Type: new Abstract: Parsing chemical reaction diagrams from scientific literature is challenging due to heterogeneous layouts, intertwined visual elements, and the difficul

OralAgent: Integrating Reasoning, Tools, and Knowledge for Interactive Dental Image Analysis

arXiv:2605.27378v1 Announce Type: new Abstract: Dental image analysis plays a pivotal role in supporting accurate diagnosis and treatment planning in oral healthcare. Although recent advances have pro

PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraft

arXiv:2605.27762v1 Announce Type: new Abstract: We present PEAM, a Parametric Embodied Agent Memory framework in Minecraft that transforms agent memory from inference-time retrieval into parameter-res

Personal Visual Memory from Explicit and Implicit Evidence

arXiv:2605.28806v1 Announce Type: cross Abstract: Long-term memory is increasingly important for personalized AI agents, yet existing benchmarks and methods remain largely text-centric. Even when imag

PhAME: Phenotype-Aware Molecular Editing via Latent Diffusion

arXiv:2605.28226v1 Announce Type: new Abstract: Small-molecule drug discovery requires simultaneous optimization of numerous properties of candidate molecules. These properties can be investigated thr

researcharxiv-cs-lg

PrimitiveVLA: Learning Reusable Motion Primitives for Efficient and Generalizable Robotic Manipulation

arXiv:2605.28634v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models offer a promising paradigm for generalist robotic policies, yet their adaptation is hindered by data inefficiency an

tutorialsarxiv-cs-ro

RMPL: Relation-aware Multi-task Progressive Learning with Stage-wise Training for Multimedia Event Extraction

arXiv:2602.13748v2 Announce Type: replace Abstract: Multimedia Event Extraction (MEE) aims to identify events and their arguments from documents that contain both text and images. It requires groundin

Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directions

arXiv:2604.08304v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) extends large language models (LLMs) with external knowledge, but this access path also introduces securi

agentsarxiv-cs-ai

Self-Supervised Online Robot-Agnostic Traversability Estimation for Open-World Environments

arXiv:2605.28442v1 Announce Type: cross Abstract: Self-supervised online traversability estimation enables robots to continuously learn from unlabeled open-world experiences and adapt their navigation

Tabero: Learning Gentle Manipulation with Closed-Loop Force Feedback from Vision, Touch, and Language

model-releasesarxiv-cs-ro

arXiv:2605.27886v1 Announce Type: new Abstract: Tactile sensing is essential for robots to achieve human-like gentle manipulation. However, existing Vision-Language-Action (VLA) models struggle to exp

The Point, the Vision and the Text: Does Point Cloud Boost Spatial Reasoning of Large Language Models? A Bias-Controlled Study

arXiv:2504.04540v2 Announce Type: replace-cross Abstract: 3D Large Language Models (LLMs) leveraging spatial information in point clouds for 3D spatial reasoning attract great attention. Despite some

The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation

arXiv:2601.17737v3 Announce Type: replace-cross Abstract: Recent advances in video generation have produced models capable of synthesizing stunning visual content from simple text prompts. However, th

Hardware

Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigation

arXiv:2605.27582v1 Announce Type: cross Abstract: Embodied navigation requires an agent to map language and visual observations to a stream of spatial actions that drive a real robot through environme

hardwarearxiv-cs-cv

VLA-Hijack: A Transferable Patch Attack against Vision-Language-Action Models via Visual Proprioception Hijacking

arXiv:2605.28083v1 Announce Type: new Abstract: While Vision-Language-Action (VLA) models have emerged as powerful generalist policies, their severe vulnerability to adversarial patches significantly

safetyarxiv-cs-cv

Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?

arXiv:2604.02028v2 Announce Type: replace Abstract: Diffusion models have become a standard approach for generative modeling in continuous domains, yet their application to discrete data remains chall

tutorialsarxiv-cs-cl

AD-H: Language-guided Autonomous Driving with Hierarchical Agents

arXiv:2406.03474v2 Announce Type: replace Abstract: Language-guided autonomous driving requires bridging a large abstraction gap between high-level natural-language instructions and low-level vehicle

agentsarxiv-cs-cv

BioFact-MoE: Biologically Factorized Mixture of Experts for Vision-Language Prognostic Modeling in Hepatocellular Carcinoma

arXiv:2605.26376v1 Announce Type: cross Abstract: Hepatocellular carcinoma (HCC) is biologically heterogeneous, shaped by the interplay between hepatic functional reserve and tumor-related oncologic f

tutorialsarxiv-cs-ai

ChartAct: A Benchmark for Dynamic Chart Understanding

arXiv:2605.26994v1 Announce Type: new Abstract: Charts are widely used to present complex data for analysis and decision making. Existing chart understanding benchmarks mainly focus on static charts,

Design First, Code Later: Aesthetically Pleasing Template-Free Slides Generation

arXiv:2605.26451v1 Announce Type: cross Abstract: Producing presentation slides automatically entails coordinating narrative structure with page-level graphic design under strict spatial constraints.

agentsarxiv-cs-cv

Doc-CoB: Enhancing Document Understanding with Visual Chain-of-Boxes Reasoning

arXiv:2505.18603v2 Announce Type: replace Abstract: Document understanding aims to perform question answering and information extraction over document images, where the visual content is highly inform

researcharxiv-cs-ai

EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy

arXiv:2605.24456v2 Announce Type: replace Abstract: Humans constantly reason about 3D proximity, the relations between their body and surrounding objects, to guide perception and action in daily life.

Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints

arXiv:2603.17685v3 Announce Type: replace Abstract: Balancing policy expressiveness with the exploration-exploitation trade-off is a core challenge in online Reinforcement Learning (RL). While Stochas

safetyarxiv-cs-lg

I2PRef: Image-Driven Point Completion with Iterative Refinement

arXiv:2605.26914v1 Announce Type: new Abstract: We present an image-conditioned point cloud completion approach that treats images as the primary geometric source rather than a secondary guide. To thi

tutorialsarxiv-cs-cv

LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV

arXiv:2605.26244v1 Announce Type: new Abstract: Audio-visual generation is rapidly advancing from short clips to minute-long content, while existing evaluation protocols remain largely confined to sho

MetaSICL: Adapting Audiroty LLM via Meta Speech In-Context Learning

arXiv:2601.18904v2 Announce Type: replace-cross Abstract: Auditory Large Language Models (LLMs) have demonstrated strong performance across a wide range of speech and audio understanding tasks. Nevert

researcharxiv-cs-ai

Applications

No Data? No Problem: Robust Vision-Tabular Learning with Missing Values

arXiv:2512.19602v2 Announce Type: replace Abstract: Large-scale medical biobanks provide imaging data complemented by extensive tabular information, such as clinical measurements or demographics. Howe

applicationsarxiv-cs-cv

O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding

arXiv:2605.26584v1 Announce Type: new Abstract: Omnimodal large language models enable unified audio video understanding, but long joint token sequences make inference costly, and existing benchmarks

OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants

arXiv:2605.26485v1 Announce Type: cross Abstract: We introduce OmniInteract, a streaming benchmark for real-time omnimodal large language models evaluated through native online inference over audio-vi

'PhyWorldBench': A Comprehensive Evaluation of Physical Realism in Text-to-Video Models

arXiv:2507.13428v3 Announce Type: replace-cross Abstract: Video generation models have achieved remarkable progress in creating high-quality, photorealistic content. However, their ability to accurate

PitchBench: Measuring Pitch Hearing in Audio-Language Models

arXiv:2605.26176v1 Announce Type: cross Abstract: Audio-language models (ALMs) are increasingly used in real-world applications that require understanding music, from music tutoring and transcription

Reconstructing Multi-Scale Physical Fields from Extremely Sparse Measurements with an Autoencoder-Diffusion Cascade

arXiv:2512.01572v3 Announce Type: replace-cross Abstract: Extreme sensor sparsity makes full-field reconstruction a fundamentally ill-posed problem in scientific sensing,where the goal is to infer phy

researcharxiv-cs-ai

Self-signals Driven Multi-LLM Debate for Efficient and Accurate Reasoning

arXiv:2510.06843v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have exhibited impressive capabilities across diverse application domains. Recent work has explored Multi-LLM Age

agentsarxiv-cs-ai

The Rescue Effect: Spatio-Semantic Early Exit Bypasses Quantization Collapse in CLIP

arXiv:2605.26415v1 Announce Type: cross Abstract: Deploying Vision-Language Models on resource-constrained hardware typically requires INT8 quantization, but in joint-embedding architectures such as C

safetyarxiv-cs-ai

Applications

Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance

arXiv:2605.24203v1 Announce Type: new Abstract: Vision-language-action (VLA) models have shown strong potential for generalist robot manipulation, yet they remain limited by insufficient spatial reaso

applicationsarxiv-cs-ro

Agent-Centric Social Trajectory Prediction: A Free Energy Principle Perspective

arXiv:2605.25748v1 Announce Type: new Abstract: Trajectory prediction methods have demonstrated remarkable capabilities in capturing complex motion patterns. However, existing methods rely on global s

safetyarxiv-cs-ai

AVBench: Human-Aligned and Automated Evaluation Benchmark for Audio-Video Generative Models

arXiv:2605.24652v1 Announce Type: new Abstract: Rapid advances in audio-video (AV) generation have enabled high-fidelity synthesis with synchronized sound, particularly for human-related scenarios inv

Double Triangle Annotation: A Scalable Human-in-the-Loop Framework for High-Precision Historical Document Annotation

arXiv:2605.25781v1 Announce Type: new Abstract: Evaluating structured-information extraction from historical documents at scale requires high-precision ground-truth annotations, yet traditional manual