Search: “multimodal”

Large-Scale AI and Foundation Models for Neuroscience: A Comprehensive Review

arXiv:2510.16658v3 Announce Type: replace Abstract: The development of large-scale artificial intelligence (AI) models is influencing neuroscience research by enabling end-to-end learning from raw bra

researcharxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

MATNet: Multi-Level Fusion Transformer-Based Model for Day-Ahead PV Generation Forecasting

DGX agent

arXiv:2306.10356v3 Announce Type: replace-cross Abstract: Accurate forecasting of renewable generation is crucial to facilitate the integration of Renewable Energy Sources into the power system. Focus

model-releasesarxiv-cs-ai

29 May 2026

Mitigating Hallucination in Vision-Language Models through Barrier-Regulated Adaptive Closed-form Steering

arXiv:2605.29881v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) often hallucinate objects that are not present in the input image, largely because visual grounding weakens as de

MMTM: Tri-Modal Topic Modeling for Long-Form Video via Similarity-Gated Fusion

arXiv:2605.29765v1 Announce Type: new Abstract: We introduce MMTM, a modular pipeline for topic discovery in long-form video that integrates speech recognition, audio and visual embeddings, and BERTop

RAISE: RAG Design as an Architecture Search Problem

arXiv:2605.30029v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) systems expose numerous design choices spanning query rewriting, chunking, retrieval depth, reranking, and context

Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

arXiv:2605.29358v1 Announce Type: new Abstract: We demonstrate that sparse autoencoders can extract interpretable features from Claude 3 Sonnet, a production-scale language model, addressing the open

Solving Integer Linear Programming with Parallel Tempering

arXiv:2605.29366v1 Announce Type: new Abstract: Integer Linear Programming (ILP) serves as a versatile framework for modeling a wide range of combinatorial optimization problems, typically addressed b

Tiny but Trusted: Efficient Vision-Language Reasoning for Time-Series Anomaly Detection

arXiv:2605.30344v1 Announce Type: new Abstract: Recent advances in Vision-Language Models (VLMs) have achieved impressive performance across many tasks, yet prior studies report unsatisfactory perform

VideoFDB: Evaluating Full-Duplex Vision-Speech Capabilities in Conversational Agents

arXiv:2605.30256v1 Announce Type: cross Abstract: Natural human conversation is full-duplex and audio-visual: people simultaneously speak and listen while continuously interpreting and producing nonve

Adaptive Temporal Gating of Longitudinal Magnetic Resonance Imaging for Alzheimer's Prediction

arXiv:2605.28397v1 Announce Type: new Abstract: Predicting conversion from Mild Cognitive Impairment (MCI) to Alzheimer's Disease (AD) is critical for early intervention. Current deep learning paradig

researcharxiv-cs-cv

Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring

arXiv:2502.05242v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are becoming increasingly capable, but the mechanisms of their thinking and decision-making processes remain uncl

Local Ai

CIVIC: End-to-End Sequence Compactness for Efficient Vision-Language Models

arXiv:2605.28115v1 Announce Type: new Abstract: Vision-Language Models (VLMs) face severe memory and latency bottlenecks due to high-resolution visual tokens. While current token reduction methods the

local-aiarxiv-cs-ai

CogPortrait: Fine-Grained Eye-Region Control in Portrait Animation via Hierarchical Agent Planning

arXiv:2605.28056v1 Announce Type: new Abstract: Portrait animation methods have achieved substantial visual quality and lip synchronization, but fine-grained manipulation of the eye region still faces

CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification

arXiv:2508.21046v3 Announce Type: replace Abstract: Recent Vision-Language-Action (VLA) models built on pre-trained Vision-Language Models (VLMs) require extensive post-training, resulting in high com

CPPO: Contrastive Perception Policy Optimization for VLM Agents

arXiv:2601.00501v2 Announce Type: replace Abstract: We introduce CPPO, a Contrastive Perception Policy Optimization method for finetuning vision--language models (VLMs). Reliable perception is a core

safetyarxiv-cs-cv

Cross-Modal Action Recognition in Egocentric Video Using Mamba: Integrating RGB and Hand Skeleton Streams via CLS Token Fusion Strategies

arXiv:2605.24302v2 Announce Type: replace Abstract: Egocentric action recognition is a challenging task due to erratic camera motion, frequent hand occlusion, and the difficulty of maintaining consist

researcharxiv-cs-cv

Cyberbullying Governance on Social Media: A Unified Framework from Content Identification to Intervention

arXiv:2605.27584v1 Announce Type: new Abstract: The proliferation of social media platforms and online communities has inadvertently catalyzed the spread of cyberbullying, hate speech, and other forms

safetyarxiv-cs-ai

Do LLMs Build World Models From Text? A Multilingual Diagnostic of Spatial Reasoning

arXiv:2605.28277v1 Announce Type: new Abstract: Whether large language models (LLMs) construct internal spatial world models from pure-text descriptions remains contested, and whether such capabilitie

ForestHG-Trace: Traceable Long-Horizon Ecological Reasoning over Large-Scale Forest Scenes

arXiv:2605.27590v1 Announce Type: new Abstract: Remote sensing question answering (RS-QA) often requires more than direct semantic prediction, especially in large-scale forest scenes where ecological

From Pixels to Words -- Towards Native One-Vision Models at Scale

arXiv:2605.28820v1 Announce Type: new Abstract: Current vision-language models (VLMs) typically stitch together separate image encoders and language decoders via multi-stage alignment, a modular frame

safetyarxiv-cs-cv

Hierarchical Synthetic Tabular Data Generation: A Hybrid Top-Down and Bottom-Up Framework

arXiv:2605.28198v1 Announce Type: new Abstract: Existing approaches for synthetic tabular data generation are based on either purely generative models or LLMs, both of which struggle with data heterog

safetyarxiv-cs-lg

Latent-Conditioned Parameterized Quantum Circuits as Universal Approximators for Distributions over Quantum States

arXiv:2605.28690v1 Announce Type: cross Abstract: Many applications in quantum simulation, quantum chemistry, and quantum machine learning require not a single quantum state but an ensemble of states

MACReD: A Multi-Agent Collaborative Reasoning Framework for Reaction Diagram Parsing

arXiv:2605.28077v1 Announce Type: new Abstract: Parsing chemical reaction diagrams from scientific literature is challenging due to heterogeneous layouts, intertwined visual elements, and the difficul

OralAgent: Integrating Reasoning, Tools, and Knowledge for Interactive Dental Image Analysis

arXiv:2605.27378v1 Announce Type: new Abstract: Dental image analysis plays a pivotal role in supporting accurate diagnosis and treatment planning in oral healthcare. Although recent advances have pro

PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraft

arXiv:2605.27762v1 Announce Type: new Abstract: We present PEAM, a Parametric Embodied Agent Memory framework in Minecraft that transforms agent memory from inference-time retrieval into parameter-res

Personal Visual Memory from Explicit and Implicit Evidence

arXiv:2605.28806v1 Announce Type: cross Abstract: Long-term memory is increasingly important for personalized AI agents, yet existing benchmarks and methods remain largely text-centric. Even when imag

PhAME: Phenotype-Aware Molecular Editing via Latent Diffusion

arXiv:2605.28226v1 Announce Type: new Abstract: Small-molecule drug discovery requires simultaneous optimization of numerous properties of candidate molecules. These properties can be investigated thr

PrimitiveVLA: Learning Reusable Motion Primitives for Efficient and Generalizable Robotic Manipulation

arXiv:2605.28634v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models offer a promising paradigm for generalist robotic policies, yet their adaptation is hindered by data inefficiency an

tutorialsarxiv-cs-ro

RMPL: Relation-aware Multi-task Progressive Learning with Stage-wise Training for Multimedia Event Extraction

arXiv:2602.13748v2 Announce Type: replace Abstract: Multimedia Event Extraction (MEE) aims to identify events and their arguments from documents that contain both text and images. It requires groundin

Agents

Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directions

arXiv:2604.08304v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) extends large language models (LLMs) with external knowledge, but this access path also introduces securi

agentsarxiv-cs-ai

Self-Supervised Online Robot-Agnostic Traversability Estimation for Open-World Environments

arXiv:2605.28442v1 Announce Type: cross Abstract: Self-supervised online traversability estimation enables robots to continuously learn from unlabeled open-world experiences and adapt their navigation

Tabero: Learning Gentle Manipulation with Closed-Loop Force Feedback from Vision, Touch, and Language

model-releasesarxiv-cs-ro

arXiv:2605.27886v1 Announce Type: new Abstract: Tactile sensing is essential for robots to achieve human-like gentle manipulation. However, existing Vision-Language-Action (VLA) models struggle to exp

The Point, the Vision and the Text: Does Point Cloud Boost Spatial Reasoning of Large Language Models? A Bias-Controlled Study

arXiv:2504.04540v2 Announce Type: replace-cross Abstract: 3D Large Language Models (LLMs) leveraging spatial information in point clouds for 3D spatial reasoning attract great attention. Despite some

The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation

arXiv:2601.17737v3 Announce Type: replace-cross Abstract: Recent advances in video generation have produced models capable of synthesizing stunning visual content from simple text prompts. However, th

Hardware

Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigation

arXiv:2605.27582v1 Announce Type: cross Abstract: Embodied navigation requires an agent to map language and visual observations to a stream of spatial actions that drive a real robot through environme

hardwarearxiv-cs-cv

VLA-Hijack: A Transferable Patch Attack against Vision-Language-Action Models via Visual Proprioception Hijacking

arXiv:2605.28083v1 Announce Type: new Abstract: While Vision-Language-Action (VLA) models have emerged as powerful generalist policies, their severe vulnerability to adversarial patches significantly

safetyarxiv-cs-cv

Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?

arXiv:2604.02028v2 Announce Type: replace Abstract: Diffusion models have become a standard approach for generative modeling in continuous domains, yet their application to discrete data remains chall

tutorialsarxiv-cs-cl

Agents

AD-H: Language-guided Autonomous Driving with Hierarchical Agents

arXiv:2406.03474v2 Announce Type: replace Abstract: Language-guided autonomous driving requires bridging a large abstraction gap between high-level natural-language instructions and low-level vehicle

agentsarxiv-cs-cv

BioFact-MoE: Biologically Factorized Mixture of Experts for Vision-Language Prognostic Modeling in Hepatocellular Carcinoma

arXiv:2605.26376v1 Announce Type: cross Abstract: Hepatocellular carcinoma (HCC) is biologically heterogeneous, shaped by the interplay between hepatic functional reserve and tumor-related oncologic f

tutorialsarxiv-cs-ai

ChartAct: A Benchmark for Dynamic Chart Understanding

arXiv:2605.26994v1 Announce Type: new Abstract: Charts are widely used to present complex data for analysis and decision making. Existing chart understanding benchmarks mainly focus on static charts,

Agents

Design First, Code Later: Aesthetically Pleasing Template-Free Slides Generation

arXiv:2605.26451v1 Announce Type: cross Abstract: Producing presentation slides automatically entails coordinating narrative structure with page-level graphic design under strict spatial constraints.

agentsarxiv-cs-cv

Doc-CoB: Enhancing Document Understanding with Visual Chain-of-Boxes Reasoning

arXiv:2505.18603v2 Announce Type: replace Abstract: Document understanding aims to perform question answering and information extraction over document images, where the visual content is highly inform

researcharxiv-cs-ai

EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy

arXiv:2605.24456v2 Announce Type: replace Abstract: Humans constantly reason about 3D proximity, the relations between their body and surrounding objects, to guide perception and action in daily life.

Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints

arXiv:2603.17685v3 Announce Type: replace Abstract: Balancing policy expressiveness with the exploration-exploitation trade-off is a core challenge in online Reinforcement Learning (RL). While Stochas

safetyarxiv-cs-lg

I2PRef: Image-Driven Point Completion with Iterative Refinement

arXiv:2605.26914v1 Announce Type: new Abstract: We present an image-conditioned point cloud completion approach that treats images as the primary geometric source rather than a secondary guide. To thi

tutorialsarxiv-cs-cv

LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV

arXiv:2605.26244v1 Announce Type: new Abstract: Audio-visual generation is rapidly advancing from short clips to minute-long content, while existing evaluation protocols remain largely confined to sho

MetaSICL: Adapting Audiroty LLM via Meta Speech In-Context Learning

arXiv:2601.18904v2 Announce Type: replace-cross Abstract: Auditory Large Language Models (LLMs) have demonstrated strong performance across a wide range of speech and audio understanding tasks. Nevert

researcharxiv-cs-ai

Applications

No Data? No Problem: Robust Vision-Tabular Learning with Missing Values

arXiv:2512.19602v2 Announce Type: replace Abstract: Large-scale medical biobanks provide imaging data complemented by extensive tabular information, such as clinical measurements or demographics. Howe

applicationsarxiv-cs-cv