Search: “evaluation”

Digital Twins as Synthetic Controls in Single-Arm Trials

arXiv:2605.12832v1 Announce Type: cross Abstract: Single-arm trials are an important study design for evaluating drug efficacy and safety without enrolling patients into a control arm. Although they d

Local Ai

Discovery of Hidden Miscalibration Regimes

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2605.13484v1 Announce Type: cross Abstract: Calibration is commonly evaluated by comparing model confidence with its empirical correctness, implicitly treating reliability as a function of the c

local-aiarxiv-cs-ai

14 May 2026

How Well Do Large-Scale Chemical Language Models Transfer to Downstream Tasks?

model-releasesarxiv-cs-lg

arXiv:2602.11618v4 Announce Type: replace Abstract: Chemical Language Models (CLMs) pre-trained on large scale molecular data are widely used for molecular property prediction. However, the common bel

Applications

MILM: Large Language Models for Multimodal Irregular Time Series with Informative Sampling

arXiv:2605.13711v1 Announce Type: new Abstract: Multimodal irregular time series (MITS) consist of asynchronous and irregularly sampled observations from heterogeneous numerical and textual channels.

applicationsarxiv-cs-lg

No One Knows the State of the Art in Geospatial Foundation Models

arXiv:2605.12678v1 Announce Type: new Abstract: Geospatial foundation models (GFMs) have been proposed as generalizable backbones for disaster response, land-cover mapping, food-security monitoring, a

ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimization

arXiv:2605.12667v1 Announce Type: cross Abstract: The alignment of Large Language Models (LLMs) utilizes Reinforcement Learning from AI Feedback (RLAIF) for non-verifiable domains such as long-form qu

PersonalAlign: Hierarchical Implicit Intent Alignment for Personalized GUI Agent with Long-Term User-Centric Records

arXiv:2601.09636v2 Announce Type: replace Abstract: While GUI agents have shown strong performance under explicit and completion instructions, real-world deployment requires aligning with users' more

PhysEditBench: A Protocol-Conditioned Benchmark for Dense Physical-Map Prediction with Image Editors

arXiv:2605.13493v1 Announce Type: new Abstract: Can general-purpose image editors predict physical maps from a single RGB image? General-purpose image editors differ from standard task-specific dense-

PreFIQs: Face Image Quality Is What Survives Pruning

arXiv:2605.13396v1 Announce Type: new Abstract: Face Image Quality Assessment (FIQA) evaluates the utility of a face image for automated face recognition (FR) systems. In this work, we propose PreFIQs

Quantitative Certification of Agentic Tool Selection

arXiv:2510.03992v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed in agentic systems, where a fundamental task is mapping user intents to relevant extern

Qwen-Image-VAE-2.0 Technical Report

arXiv:2605.13565v1 Announce Type: new Abstract: We present Qwen-Image-VAE-2.0, a suite of high-compression Variational Autoencoders (VAEs) that achieve significant advances in both reconstruction fide

Tutorials

Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education

arXiv:2605.12988v1 Announce Type: new Abstract: Students learning algorithms often need support as they interpret traces, debug reasoning errors, and apply procedures across unfamiliar problem instanc

tutorialsarxiv-cs-ai

Runtime Monitoring of Perception-Based Autonomous Systems via Embedding Temporal Logic

arXiv:2605.12651v1 Announce Type: new Abstract: Runtime monitoring of autonomous systems traditionally relies on mapping continuous sensor observations to discrete logical propositions defined over lo

SpatialReward: Bridging the Perception Gap in Online RL for Image Editing via Explicit Spatial Reasoning

arXiv:2602.07458v4 Announce Type: replace Abstract: Online Reinforcement Learning (RL) offers a promising avenue for complex image editing but is currently constrained by the scarcity of reliable and

StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos

arXiv:2512.01707v3 Announce Type: replace-cross Abstract: Streaming video understanding requires models not only to process temporally incoming frames, but also to anticipate user intention for realis

Sustaining AI safety: Control-theoretic external impossibility, intrinsic necessity, and structural requirements

arXiv:2605.12963v1 Announce Type: new Abstract: As AI systems become increasingly capable, safety strategies must be evaluated not only by how much they reduce present risk, but by whether they could

Test-time Offline Reinforcement Learning on Goal-related Experience

arXiv:2507.18809v2 Announce Type: replace Abstract: Foundation models compress a large amount of information in a single, large neural network, which can then be queried for individual tasks. There ar

AgentDisCo: Towards Disentanglement and Collaboration in Open-ended Deep Research Agents

arXiv:2605.11732v1 Announce Type: cross Abstract: In this paper, we present AgentDisCo, a novel Disentangled and Collaborative agentic architecture that formulates deep research as an adversarial opti

Clarity: The Flexibility-Interpretability Trade-Off in Sparsity-aware Concept Bottleneck Models

arXiv:2601.21944v2 Announce Type: replace Abstract: The widespread adoption of deep learning models in computer vision has intensified concerns about interpretability. Despite strong performance, thes

Agents

DreamAvoid: Critical-Phase Test-Time Dreaming to Avoid Failures in VLA Policies

arXiv:2605.11750v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models are often brittle in fine-grained manipulation, where minor action errors during the critical phases can rapidly e

agentsarxiv-cs-cl

FLAME: A New Dataset on FLemish Accounts of Momentary Experiences

arXiv:2504.14707v3 Announce Type: replace Abstract: We introduce FLAME (FLemish Accounts of Momentary Experiences), a new corpus of nearly 25,000 daily personal narratives in Belgian-Dutch (Flemish),

Generative AI for Visualizing Highway Construction Hazards Through Synthetic Images and Temporal Sequences

arXiv:2605.11276v1 Announce Type: new Abstract: Highway construction workers face a high risk of serious injury or death. Image-based training materials depicting hazardous scenarios are essential for

safetyarxiv-cs-cv

HE-SNR: Uncovering Latent Logic via Entropy for Guiding Mid-Training on SWE-bench

arXiv:2601.20255v2 Announce Type: replace-cross Abstract: SWE-bench has emerged as the premier benchmark for evaluating Large Language Models on complex software engineering tasks. While these capabil

Applications

Interpretability Can Be Actionable

arXiv:2605.11161v1 Announce Type: new Abstract: Interpretability aims to explain the behavior of deep neural networks. Despite rapid growth, there is mounting concern that much of this work has not tr

applicationsarxiv-cs-lg

LiBrA-Net: Lie-Algebraic Bilateral Affine Fields for Real-Time 4K Video Dehazing

arXiv:2605.11508v1 Announce Type: new Abstract: Currently, there is a gap in the field of ultra-high-definition (UHD) video dehazing due to the lack of a benchmark for evaluation. Furthermore, existin

Nautilus: From One Prompt to Plug-and-Play Robot Learning

model-releasesarxiv-cs-ro

arXiv:2605.11665v1 Announce Type: new Abstract: Robot learning research is fragmented across policy families, benchmark suites, and real robots; each implementation is entangled with the others in a c

OUI as a Structural Observable: Towards an Activation-Centric View of Neural Network Training

arXiv:2605.11570v1 Announce Type: new Abstract: Activation functions are what make deep networks expressive: without them, the model collapses to a linear map. Yet we still evaluate training mostly fr

researcharxiv-cs-lg

Premover: Fast Vision-Language-Action Control by Acting Before Instructions Are Complete

model-releasesarxiv-cs-ro

arXiv:2605.12160v1 Announce Type: new Abstract: Vision-Language-Action (VLA) policies are typically evaluated as if the user had finished typing or speaking before the robot begins acting. In real dep

Tutorials

Sharpen Your Flow: Sharpness-Aware Sampling for Flow Matching

arXiv:2605.11547v1 Announce Type: new Abstract: Flow matching models generate samples by numerically integrating a learned velocity field, with each integration step requiring a neural network evaluat

tutorialsarxiv-cs-lg

UHR-Micro: Diagnosing and Mitigating the Resolution Illusion in Earth Observation VLMs

arXiv:2605.12237v1 Announce Type: new Abstract: Vision-Language Models (VLMs) increasingly operate on ultra-high-resolution (UHR) Earth observation imagery, yet they remain vulnerable to a severe scal

XWOD: A Real-World Benchmark for Object Detection under Extreme Weather Conditions

arXiv:2605.11521v1 Announce Type: new Abstract: Autonomous driving and intelligent transportation systems remain vulnerable under extreme weather. The U.S. Federal Highway Administration reports that

A Paired Point-of-Care Ultrasound Dataset for Image Quality Enhancement and Benchmarking via a cGAN Baseline

arXiv:2605.08282v1 Announce Type: cross Abstract: Purpose: We aim to enhance the image quality of point-of-care ultrasound (POCUS) devices using deep learning and a novel paired dataset of POCUS and h

researcharxiv-cs-ai

A Scalable Entity-Based Framework for Auditing Bias in LLMs

arXiv:2601.12374v2 Announce Type: replace-cross Abstract: Existing approaches to bias evaluation in large language models (LLMs) trade ecological validity for statistical control, relying either on ar

Adaptive DNN Partitioning and Offloading in Heterogeneous Edge-Cloud Continuum

arXiv:2605.09623v1 Announce Type: cross Abstract: In recent years, the use of artificial intelligence on resource-constrained IoT devices has grown significantly. However, existing approaches to DNN p

researcharxiv-cs-ai

Advanced Tumor Segmentation in PET/CT Imaging: A Training Strategy Study with nnU-Net for AutoPET III

arXiv:2605.08161v1 Announce Type: new Abstract: Tumor segmentation in whole-body PET/CT imaging is crucial for precise disease evaluation and treatment planning. However, it remains challenging due to

researcharxiv-cs-cv

AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators

arXiv:2605.08647v1 Announce Type: cross Abstract: Multi-agent systems achieve state-of-the-art outcomes through peer collaboration. However, when an agent in the pipeline silently drops a constraint,

Aligning Agents via Planning: A Benchmark for Trajectory-Level Reward Modeling

arXiv:2604.08178v2 Announce Type: replace Abstract: In classical Reinforcement Learning from Human Feedback (RLHF), Reward Models (RMs) serve as the fundamental signal provider for model alignment. As

Ambig-DS: A Benchmark for Task-Framing Ambiguity in Data-Science Agents

arXiv:2605.09698v1 Announce Type: new Abstract: As data-science agents shift from co-pilots to auto-pilots, silent misframing becomes a critical failure mode. Agents quietly commit to plausible but un

An Empirical Analysis of Calibration and Selective Prediction in Multimodal Clinical Condition Classification

arXiv:2603.02719v2 Announce Type: replace Abstract: As artificial intelligence systems move toward clinical deployment, ensuring reliable prediction behavior is fundamental for safety-critical decisio

Assessing Trustworthiness of AI Training Dataset using Subjective Logic -- A Use Case on Bias

arXiv:2508.13813v2 Announce Type: replace-cross Abstract: As AI systems increasingly rely on training data, assessing dataset trustworthiness has become critical, particularly for properties like fair

ASTRA-QA: A Benchmark for Abstract Question Answering over Documents

arXiv:2605.10168v1 Announce Type: new Abstract: Document-based question answering (QA) increasingly includes abstract questions that require synthesizing scattered information from long documents or a

Applications

Behavior-Centric Extraction of Scenarios from Highway Traffic Data and their Domain-Knowledge-Guided Clustering using CVQ-VAE

arXiv:2603.16964v2 Announce Type: replace Abstract: Approval of ADS depends on evaluating its behavior within representative real-world traffic scenarios. A common way to obtain such scenarios is to e

applicationsarxiv-cs-cv

Benchmarking Safety Risks of Knowledge-Intensive Reasoning under Malicious Knowledge Editing

arXiv:2605.10146v1 Announce Type: new Abstract: Large language models (LLMs) increasingly rely on knowledge editing to support knowledge-intensive reasoning, but this flexibility also introduces criti

ChaosNetBench: Benchmarking Spatio-Temporal Graph Neural Networks on Chaotic Lattice Dynamics

arXiv:2605.09676v1 Announce Type: cross Abstract: Spatio-temporal graph neural networks (STGNNs) are widely used for short-term forecasting in dynamic physical systems such as traffic and weather. How

Code Mixologist : A Practitioner's Guide to Building Code-Mixed LLMs

arXiv:2602.11181v2 Announce Type: replace Abstract: Code-mixing and code-switching (CSW) remain challenging phenomena for large language models (LLMs). Despite recent advances in multilingual modeling

safetyarxiv-cs-cl

CUDABeaver: Benchmarking LLM-Based Automated CUDA Debugging

model-releasesarxiv-cs-lg

arXiv:2605.08455v1 Announce Type: new Abstract: Debugging CUDA programs has long been challenging because failures often arise from subtle interactions among hardware behavior, compiler decisions, mem

Agents

DRIVE-C: A Controlled Corruption Dataset for Autonomous Driving

arXiv:2605.09774v1 Announce Type: new Abstract: DRIVE-C is a controlled corruption dataset designed to evaluate visual perception robustness in autonomous driving systems. It is built from real-world

agentsarxiv-cs-cv

EpiGraph: A Knowledge Graph and Benchmark for Evidence-Intensive Reasoning in Epilepsy

arXiv:2605.09505v1 Announce Type: new Abstract: Epilepsy diagnosis and treatment require evidence-intensive reasoning across heterogeneous clinical knowledge, including biosignal patterns, genetic mec