Search: “multimodal”

PERL: Parameter Efficient Reasoning in CLIP Latent Space

arXiv:2605.18464v1 Announce Type: new Abstract: Contrastively trained vision-language models such as CLIP provide strong zero-shot transfer by aligning images and text in a shared embedding space. How

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Prediction of Challenging Behaviors Associated with Profound Autism in a Classroom Setting Using Wearable Sensors

DGX agent

arXiv:2605.17618v1 Announce Type: new Abstract: Autism Spectrum Disorder (ASD) is characterized by challenges with social interaction and communication and by restricted or repetitive patterns of thou

safetyarxiv-cs-ai

19 May 2026

Agents

Qumus: Realization of An Embodied AI Quantum Material Experimentalist

arXiv:2605.18407v1 Announce Type: cross Abstract: While modern Large Language Models (LLMs) and agentic artificial intelligence (AI) have demonstrated transformative capabilities in digital domains, t

agentsarxiv-cs-ai

Local Ai

Robo-Cortex: A Self-Evolving Embodied Agent via Dual-Grain Cognitive Memory and Autonomous Knowledge Induction

arXiv:2605.18729v1 Announce Type: cross Abstract: The ability to navigate and interact with complex environments is central to real-world embodied agents, yet navigation in unseen environments remains

local-aiarxiv-cs-cv

SIREM: Speech-Informed MRI Reconstruction with Learned Sampling

model-releasesarxiv-cs-cl

arXiv:2605.18221v1 Announce Type: cross Abstract: Real-time magnetic resonance imaging (rtMRI) of speech production enables non-invasive visualization of dynamic vocal-tract motion and is valuable for

SVL: Spike-based Vision-language Pretraining for Efficient 3D Open-world Understanding

arXiv:2505.17674v2 Announce Type: replace Abstract: Spiking Neural Networks (SNNs) provide an energy-efficient way to extract 3D spatio-temporal features. However, existing SNNs still exhibit a signif

Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEG

arXiv:2602.23410v3 Announce Type: replace-cross Abstract: Brain foundation models have achieved remarkable advances across a wide range of neuroscience tasks. However, most existing models are limited

Tutorials

Integrating chemical structures as treatments improves representations of microscopy images for morphological profiling

arXiv:2504.09544v3 Announce Type: replace-cross Abstract: Recent advances in self-supervised deep learning have improved our ability to quantify cellular morphological changes in high-throughput micro

tutorialsarxiv-cs-cv

PhysBrain 1.0 Technical Report

arXiv:2605.15298v1 Announce Type: cross Abstract: Vision-language-action models have advanced rapidly, but robot trajectories alone provide limited coverage for learning broad physical understanding.

UAM: A Dual-Stream Perspective on Forgetting in VLA Training

arXiv:2605.15735v1 Announce Type: cross Abstract: Vision--language--action (VLA) models are typically built by fine-tuning a pretrained vision--language model (VLM) on action data. However, we show th

Visual Compositional Tuning

arXiv:2504.21850v3 Announce Type: replace Abstract: Visual instruction tuning (VIT) datasets have grown rapidly in scale, yet the informativeness of individual training samples has largely been overlo

Action Emergence from Streaming Intent

arXiv:2605.12622v1 Announce Type: cross Abstract: We formalize action emergence as a target capability for end-to-end autonomous driving: the ability to generate physically feasible, semantically appr

Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing

arXiv:2605.12876v1 Announce Type: new Abstract: Randomized smoothing provides strong, model-agnostic robustness certificates, but existing guarantees are limited to single modalities, treating continu

safetyarxiv-cs-lg

ReTool-Video: Recursive Tool-Using Video Agents with Meta-Augmented Tool Grounding

arXiv:2605.13228v1 Announce Type: cross Abstract: Video understanding requires active evidence seeking, motivating tool-augmented video agents for temporal reasoning, cross-modal understanding, and co

Stable Attention Response for Reliable Precipitation Nowcasting

arXiv:2605.13181v1 Announce Type: cross Abstract: Precipitation nowcasting remains challenging due to the highly localized, rapidly evolving, and heterogeneous nature of atmospheric dynamics. Although

ForceFlow: Learning to Feel and Act via Contact-Driven Flow Matching

arXiv:2605.11048v1 Announce Type: new Abstract: Existing imitation learning methods enable robots to interact autonomously with the physical environment. However, contact-rich manipulation tasks remai

safetyarxiv-cs-ro

HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer

arXiv:2605.11061v1 Announce Type: new Abstract: The evolution of visual generative models has long been constrained by fragmented architectures relying on disjoint text encoders and external VAEs. In

Applications

Instruct-ICL: Instruction-Guided In-Context Learning for Post-Disaster Damage Assessment

arXiv:2605.11439v1 Announce Type: new Abstract: Rapid and accurate situational awareness is essential for effective response during natural disasters, where delays in analysis can significantly hinder

applicationsarxiv-cs-cv

When Brains Disagree: Biological Ambiguity Underlies the Challenge of Amyloid PET Synthesis from Structural MRI

arXiv:2605.11867v1 Announce Type: new Abstract: Structural MRI-to-amyloid PET synthesis has been proposed as a non-invasive alternative for amyloid assessment in Alzheimer's disease (AD). However, rep

Anchoring the Eigengap: Cross-Modal Spectral Stabilization for Sample-Efficient Representation Learning

arXiv:2605.08764v1 Announce Type: cross Abstract: Deep vision models degrade sharply in low-data regimes, particularly in medical imaging where labeled samples are scarce. We show this arises not mere

BenchCAD: A Comprehensive, Industry-Standard Benchmark for Programmatic CAD

arXiv:2605.10865v1 Announce Type: new Abstract: Industrial Computer-Aided Design (CAD) code generation requires models to produce executable parametric programs from visual or textual inputs. Beyond r

Beyond ESG Scores: Learning Dynamic Constraints for Sequential Portfolio Optimization

arXiv:2605.09310v1 Announce Type: new Abstract: ESG-aware portfolio optimization is increasingly important for sustainable capital allocation, yet most learning-based methods still operationalize ESG

safetyarxiv-cs-ai

Bridging Modalities, Spanning Time: Structured Memory for Ultra-Long Agentic Video Reasoning

arXiv:2605.08271v1 Announce Type: cross Abstract: Understanding ultra-long videos such as egocentric recordings, live streams, or surveillance footage spanning days to weeks, remains a challenge. For

dFlowGRPO: Rate-Aware Policy Optimization for Discrete Flow Models

arXiv:2605.09291v1 Announce Type: new Abstract: Discrete flow models (DFMs) are a class of flexible generative models for generating discrete data, and diffusion large language models (dLLMs) can be v

safetyarxiv-cs-lg

Free Energy Manifold: Score-Based Inference for Hybrid Bayesian Networks

arXiv:2605.09839v1 Announce Type: cross Abstract: We introduce the Free Energy Manifold (FEM), a score-trained conditional energy model specialized for inference in hybrid Bayesian networks with discr

Mixture of Experts for Recognizing Depression from Interview and Reading Tasks

arXiv:2502.20213v2 Announce Type: replace Abstract: Depression is a mental disorder and can cause a variety of symptoms, including psychological, physical, and social. Speech has been proved an object

researcharxiv-cs-lg

MOTOR-Bench: A Real-world Dataset and Multi-agent Framework for Zero-shot Human Mental State Understanding

arXiv:2605.09703v1 Announce Type: new Abstract: Understanding human mental states from natural behavior is crucial for intelligent systems in the real world. However, most current research focuses on

Nested Slice Sampling: Vectorized Nested Sampling for GPU-Accelerated Inference

model-releasesarxiv-cs-lg

arXiv:2601.23252v2 Announce Type: replace-cross Abstract: Model comparison and calibrated uncertainty quantification often require integrating over parameters, but scalable inference can be challengin

PrimeKG-CL: A Continual Graph Learning Benchmark on Evolving Biomedical Knowledge Graphs

arXiv:2605.10529v1 Announce Type: new Abstract: Biomedical knowledge graphs underwrite drug repurposing and clinical decision support, yet the upstream ontologies they depend on update on independent

Qwen-Image-2.0 Technical Report

arXiv:2605.10730v1 Announce Type: new Abstract: We present Qwen-Image-2.0, an omni-capable image generation foundation model that unifies high-fidelity generation and precise image editing within a si

Agents

SAR-RAG: ATR Visual Question Answering by Semantic Search, Retrieval, and MLLM Generation

arXiv:2602.04712v2 Announce Type: replace-cross Abstract: We present a visual-context image-retrieval-augmented generation (ImageRAG)- assisted AI agent for automatic target recognition (ATR) of synth

agentsarxiv-cs-ai

SayNext-Bench: Why Do LLMs Struggle with Next-Utterance Anticipation?

arXiv:2602.00327v2 Announce Type: replace Abstract: We explore the use of large language models (LLMs) for next-utterance anticipation in human dialogue. Despite recent advances in LLMs demonstrating

Agents

VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Driving

arXiv:2605.08830v1 Announce Type: cross Abstract: End-to-end autonomous driving requires models to understand traffic scenes, infer driving intent, and generate executable motion plans. Recent vision-

agentsarxiv-cs-ai

Bayesian Conformal Prediction as a Decision Risk Problem

arXiv:2602.03331v2 Announce Type: replace Abstract: We propose Bayesian Conformal Prediction (BCP), a framework that combines Bayesian posterior predictive distributions with PAC-style conformal risk

researcharxiv-cs-lg

ChartREG++: Towards Benchmarking and Improving Chart Referring Expression Grounding under Diverse referring clues and Multi-Target Referring

model-releasesarxiv-cs-cl

arXiv:2605.07415v1 Announce Type: cross Abstract: Referring expression grounding is a core problem in visual grounding and is widely used as a diagnostic of spatial grounding and reasoning in vision a

Fine-tuning a vision-language model for fracture-surface morphology recognition

arXiv:2605.07145v1 Announce Type: cross Abstract: Vision-language models (VLMs) have shown strong potential for scientific image understanding, but general-purpose models often lack the domain-specifi

From Standalone LLMs to Integrated Intelligence: A Survey of Compound Al Systems

arXiv:2506.04565v2 Announce Type: replace-cross Abstract: Compound AI Systems (CAIS) are an emerging paradigm that integrates large language models (LLMs) with external components, including retriever

researcharxiv-cs-cl

GRAPE: Let GRPO Supervise Query Rewriting by Ranking for Retrieval

arXiv:2509.23370v2 Announce Type: replace Abstract: The CLIP model has established itself as a cornerstone of large-scale retrieval systems. However, its performance often degrades under distributiona

InterCoG: Towards Spatially Precise Image Editing with Interleaved Chain-of-Grounding Reasoning

arXiv:2603.01586v3 Announce Type: replace Abstract: Emerging unified editing models have demonstrated strong capabilities in general object editing tasks. However, it remains a significant challenge t

PRPO: Paragraph-level Policy Optimization for Vision-Language Deepfake Detection

arXiv:2509.26272v3 Announce Type: replace Abstract: The rapid rise of synthetic media has made deepfake detection a critical challenge for online safety and trust. Progress remains constrained by the

BOOOM: Loss-Function-Agnostic Black-Box Optimization over Orthonormal Manifolds for Machine Learning and Statistical Inference

arXiv:2605.04087v1 Announce Type: cross Abstract: Optimization over the Stiefel manifold St(p,d), the set of p imes d column-orthonormal matrices, is fundamental in statistics, machine learning, and s

researcharxiv-cs-lg

7 May 2026

Audio-Visual Intelligence in Large Foundation Models

arXiv:2605.04045v1 Announce Type: new Abstract: Audio-Visual Intelligence (AVI) has emerged as a central frontier in artificial intelligence, bridging auditory and visual modalities to enable machines

Context- and Pixel-aware Large Language Model for Video Quality Assessment

arXiv:2505.16025v3 Announce Type: replace Abstract: Video quality assessment (VQA) is a challenging research topic with broad applications. Traditional hand-crafted and discriminative learning-based V

DiagramNet: An End-to-End Recognition Framework and Dataset for Non-Standard System-Level Diagrams

arXiv:2605.01338v1 Announce Type: new Abstract: System-level diagrams encode the architectural blueprint of chip design, specifying module functions, dataflows, and interface protocols. However, non-s

Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses

arXiv:2605.02900v1 Announce Type: cross Abstract: Embodied Artificial Intelligence (Embodied AI) integrates perception, cognition, planning, and interaction into agents that operate in open-world, saf

Applications

A Comprehensive Review of Fish Feeding Behavior Analysis in Aquaculture: Tasks, Techniques, and Applications

arXiv:2502.15311v3 Announce Type: replace Abstract: Fish feeding behavior analysis is a key foundation for intelligent feeding and precision aquaculture management, and plays an important role in impr

applicationsarxiv-cs-cv

5 May 2026

HeteroRAG: A Heterogeneous Retrieval-Augmented Generation Framework for Medical Vision Language Tasks

arXiv:2508.12778v2 Announce Type: replace Abstract: Medical large vision-language Models (Med-LVLMs) have shown promise in clinical applications but suffer from factual inaccuracies and unreliable out

safetyarxiv-cs-cl

5 May 2026

MedMosaic: A Challenging Large Scale Benchmark of Diverse Medical Audio