Search: “evaluation”

Prediction-Powered Inference Across Many Tasks for AI Evaluation & Social Science Research

arXiv:2605.29249v1 Announce Type: cross Abstract: Many applications require statistically valid inference across many related tasks, while using only a handful of high-quality labels per hypothesis. I

applicationsarxiv-cs-lg

29 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation

DGX agent

arXiv:2605.29430v1 Announce Type: new Abstract: Automatic speech recognition (ASR) is a core component of human--computer interaction and an increasingly important front-end for LLM-based assistants a

safetyarxiv-cs-ai

29 May 2026

A Systematic Evaluation of Retrieval-Augmented Generation and Language Models for Space Operations

arXiv:2605.27444v1 Announce Type: cross Abstract: The rapid expansion of space activities has led to an unprecedented accumulation of technical documentation, operational guidelines, and scientific li

Adaptive Cost-Efficient Evaluation for Reliable Patent Claim Generation

arXiv:2604.04295v3 Announce Type: replace Abstract: Automated patent claim validation demands low error tolerance. However, existing approaches face a rigidity-resource dilemma: lightweight encoders c

Chance-Constrained MPPI under State and Dynamic Object Prediction Uncertainty and the Evaluation of Collision Risk Calibration

arXiv:2605.28330v1 Announce Type: new Abstract: Chance-constrained Model Predictive Path Integral (MPPI) control is increasingly adopted for navigation in dynamic environments to explicitly bound coll

safetyarxiv-cs-ro

Insurance Pricing Optimization via Off-Policy Evaluation

arXiv:2605.28327v1 Announce Type: cross Abstract: Traditional insurance pricing relies on risk-based principles that ensure actuarial fairness and solvency but do not explicitly account for policyhold

safetyarxiv-cs-lg

Qwen-Image-Bench: From Generation to Creation in Text-to-Image Evaluation

arXiv:2605.28091v1 Announce Type: new Abstract: Text-to-Image generation has evolved from basic image synthesis into a frequently used core capability in professional creative workflows, where simple

RAGe: A Retrieval-Augmented Generation Evaluation Framework

arXiv:2605.27445v1 Announce Type: cross Abstract: Deploying Large Language Model (LLM) applications, particularly those relying on Retrieval-Augmented Generation (RAG), remains challenging due to high

The Importance of Being Statistically Earnest: A Critical Re-evaluation of GSM-Symbolic

arXiv:2605.28700v1 Announce Type: new Abstract: The GSM-Symbolic benchmark (Mirzadeh et al., 2025) reported consistent performance drops across 25 Large Language Models (LLMs) when tested on template-

AI-Driven Contribution Evaluation and Conflict Resolution: A Framework & Design for Group Workload Investigation

arXiv:2511.07667v2 Announce Type: replace Abstract: The equitable assessment of individual contribution in teams remains a persistent challenge, where conflict and disparity in workload can result in

safetyarxiv-cs-ai

Prospective evaluation of multimodal respiratory failure prediction: Do chest X-rays improve performance beyond EHR signals?

arXiv:2605.26255v1 Announce Type: cross Abstract: Early prediction of respiratory failure is critical for timely clinical intervention in intensive care units. Existing electronic health record (EHR)-

SL-BiLEM: Structured Learnable Behavior-in-the-Loop Epidemic Modeling for Forecasting and Policy Evaluation

arXiv:2605.26704v1 Announce Type: cross Abstract: Epidemic forecasting faces a fundamental challenge: human behavior dynamically responds to disease spread, creating feedback loops that induce distrib

safetyarxiv-cs-ai

Small Object Detection in Industrial Recycling: A New Dataset and YOLO Performance Evaluation

arXiv:2605.26884v1 Announce Type: new Abstract: In this paper, we address the problem of detecting small, dense, and overlapping objects, a major challenge in computer vision. Our focus is on reviewin

researcharxiv-cs-cv

Benchmarking Patent Embeddings: A Multi-Task Evaluation of 22 Models Across Retrieval, Classification, and Clustering

arXiv:2605.24297v1 Announce Type: cross Abstract: Which fine-tuning signals improve patent embedding models, and do gains transfer across patent landscapes? We benchmark 22 embedding models, from 22M-

From Prompt Optimization to Multi-Dimensional Credibility Evaluation: Enhancing Trustworthiness of Chinese LLM-Generated Liver MRI Reports -- with Preliminary Extension to Lung Cancer

arXiv:2510.23008v3 Announce Type: replace Abstract: Large language models (LLMs) have demonstrated promising performance in generating diagnostic conclusions from imaging findings, thereby supporting

Local Ai

Multiscale Real-Time Object Detection in the NMS-Free Era: A Comparative Performance Evaluation of YOLOv8 and YOLO26

arXiv:2605.24831v1 Announce Type: cross Abstract: Non-Maximum Suppression (NMS) remains a key post-processing step in many real-time object detection pipelines, but it can introduce latency variation

local-aiarxiv-cs-ai

Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation

arXiv:2605.24904v1 Announce Type: new Abstract: Machine-translated benchmarks are widely used to assess the multilingual capabilities of large language models (LLMs), yet translation errors in these b

RCTs & Human Uplift Studies: Methodological Challenges and Practical Solutions for Frontier AI Evaluation

arXiv:2603.11001v2 Announce Type: replace-cross Abstract: Human uplift studies, or studies that measure the effects of AI access on human performance via randomized controlled trials (RCT) or similar

applicationsarxiv-cs-ai

CapTrack: Multifaceted Evaluation of Forgetting in LLM Post-Training

arXiv:2603.06610v2 Announce Type: replace Abstract: Large language model (LLM) post-training enhances latent skills, unlocks value alignment, improves performance, and enables domain adaptation. Unfor

safetyarxiv-cs-lg

25 May 2026

How Human-Like Are Large Language Models? A Register-Aware Linguistic Evaluation Framework

arXiv:2605.23651v1 Announce Type: new Abstract: While factual correctness and task-performance have been in focus of Large Language Model (LLM) research for a long time, the fundamental question of ho

applicationsarxiv-cs-cl

25 May 2026

Adaptive RBF-KAN: A Comparative Evaluation of Dynamic Shape Parameters in Kolmogorov-Arnold Networks

model-releasesarxiv-cs-lg

arXiv:2605.21534v1 Announce Type: cross Abstract: Kolmogorov-Arnold Networks (KANs) approximate multivariate functions using learnable univariate edge functions, typically parameterized by B-spline ba

23 May 2026

Beyond Benchmark Islands: Toward Representative Trustworthiness Evaluation for Agentic AI

arXiv:2603.14987v2 Announce Type: replace Abstract: Agentic AI systems increasingly act through tool-augmented, multi-step workflows whose failures (unsafe tool use, unauthorised actions, social harm)

22 May 2026

Making the Discrete Continuous: Synthetic RAW Augmentations for Fine-Grained Evaluation of Person Detection Performance in Low Light

arXiv:2605.22455v1 Announce Type: new Abstract: Real-world deployment of AI vision models is both fueled and limited by the data available for training and testing. Real datasets are sparse and uneven

safetyarxiv-cs-cv

22 May 2026

Comparative Evaluation of Deep Learning Models for Fake Image Detection

arXiv:2605.20971v1 Announce Type: new Abstract: The growing sophistication of GAN-based image manipulation presents significant challenges for digital forensics. This study compares the performance of

safetyarxiv-cs-cv

21 May 2026

EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Models

arXiv:2510.05942v3 Announce Type: replace Abstract: We present EvalMORAAL, a transparent chain-of-thought (CoT) framework that uses two scoring methods (log-probabilities and direct ratings) plus a mo

safetyarxiv-cs-cl

21 May 2026

Real-World On-Vehicle Evaluation of Embedding-Based Anomaly Detection

arXiv:2605.19744v1 Announce Type: new Abstract: Detecting anomalies in traffic scenes is crucial for ensuring safety in autonomous driving, yet collecting representative anomalous data remains challen

20 May 2026

Towards Consistent Detection of Cognitive Distortions: LLM-Based Annotation and Dataset-Agnostic Evaluation

arXiv:2511.01482v2 Announce Type: replace Abstract: Text-based automated Cognitive Distortion detection is a challenging task due to its subjective nature, with low agreement scores observed even amon

20 May 2026

Empirical evaluation of Time Series Foundation Models for Day-ahead and Imbalance Electricity Price Forecasting in Belgium

arXiv:2605.17045v1 Announce Type: cross Abstract: Recent advances in Time Series Foundation Models (TSFMs) promise zero-shot forecasting capabilities with minimal task-specific training. While these m

researcharxiv-cs-lg

19 May 2026

Rethinking GNNs and Missing Features: Challenges, Evaluation and a Robust Solution

arXiv:2601.04855v2 Announce Type: replace-cross Abstract: Handling missing node features is a key challenge for deploying Graph Neural Networks (GNNs) in real-world domains such as healthcare and sens

19 May 2026

Systematic Evaluation of Vision Transformers for Automated Cervical Cancer Classification: Optimization, Statistical Validation, and Clinical Interpretability

arXiv:2605.17236v1 Announce Type: cross Abstract: Manual Pap smear analysis for cervical cancer screening is limited by inter-observer variability, time constraints, and restricted expert availability

19 May 2026

Evaluation of Anatomical Shape Priors in Deep Learning-Based Cardiac Multi-Compartment Segmentation

arXiv:2605.15707v1 Announce Type: cross Abstract: Whole-heart multi-compartment CT segmentation is clinically important, but standard CNNs do not explicitly enforce anatomical plausibility. Based on s

researcharxiv-cs-cv

Agents

LEAP: Trajectory-Level Evaluation of LLMs in Iterative Scientific Design

arXiv:2605.15341v1 Announce Type: cross Abstract: LLMs are increasingly deployed in autonomous laboratories, under the assumption that their domain priors and reasoning over iterative feedback let the

agentsarxiv-cs-ai

MuteBench: Modality Unavailability Tolerance Evaluation for Incomplete Multimodal Fusion

model-releasesarxiv-cs-lg

arXiv:2605.15235v1 Announce Type: new Abstract: Multimodal physiological data powers clinical AI systems from intensive care units to wearable devices, but sensors routinely fail in practice. Two fail

Reasoners or Translators? Contamination-aware Evaluation and Neuro-Symbolic Robustness in Tax Law

arXiv:2605.16052v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have significantly enhanced automated legal reasoning. Yet, it remains unclear whether their performance

applicationsarxiv-cs-ai

The OncoReach Stylet for Brachytherapy: Design Evaluation and Pilot Study

arXiv:2601.13529v2 Announce Type: replace Abstract: Cervical cancer accounts for a significant portion of the global cancer burden among women. Interstitial brachytherapy (ISBT) is a standard procedur

researcharxiv-cs-ro

GUIGuard-Bench: Toward a General Evaluation for Privacy-Preserving GUI Agents

arXiv:2601.18842v3 Announce Type: replace-cross Abstract: As GUI agents increasingly rely on screenshots to perceive and operate digital environments, they may inadvertently expose sensitive informati

14 May 2026

Tutorials

A Transfer Learning Evaluation of Deep Neural Networks for Image Classification

arXiv:2605.11989v1 Announce Type: new Abstract: Transfer learning is a machine learning technique that uses previously acquired knowledge from a source domain to enhance learning in a target domain by

tutorialsarxiv-cs-cv

13 May 2026

Interpretable EEG Microstate Discovery via Variational Deep Embedding: A Systematic Architecture Search with Multi-Quadrant Evaluation

arXiv:2605.10947v1 Announce Type: new Abstract: EEG microstate analysis segments continuous brain electrical activity into brief, quasi-stable topographic configurations that reflect discrete function

researcharxiv-cs-lg

13 May 2026

Causal Parametric Drift Simulation: A Digital Twin Framework for Classifier Robustness Evaluation

arXiv:2605.09663v1 Announce Type: cross Abstract: Machine learning classifiers in dynamic environments face concept drift -- changes in the data-generating process that degrade performance. Convention

Classification-Head Bias in Class-Level Machine Unlearning: Diagnosis, Mitigation, and Evaluation

model-releasesarxiv-cs-lg

arXiv:2605.08730v1 Announce Type: new Abstract: Class-level machine unlearning aims to remove the influence of specified classes while preserving model utility on retained classes. Existing methods ar

Do Foundation Model Embeddings Improve Cross-Country Crop Yield Generalisation? A Leave-One-Country-Out Evaluation in Sub-Saharan Africa

arXiv:2605.08113v1 Announce Type: cross Abstract: Accurate predictions of smallholder maize yields across national boundaries are critical for food security planning in sub-Saharan Africa, yet most pu

Fin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance Domain

arXiv:2605.09106v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in financial contexts, raising critical concerns about reliability, alignment, and susceptibility

General Agent Evaluation

arXiv:2602.22953v2 Announce Type: replace Abstract: General-purpose agents perform tasks in unfamiliar environments without domain-specific manual customization. Yet no study has systematically measur

H-POPE: Hierarchical Polling-based Probing Evaluation of Hallucinations in Large Vision-Language Models

arXiv:2411.04077v2 Announce Type: replace Abstract: By leveraging both texts and images, large vision language models (LVLMs) have shown significant progress in various multi-modal tasks. Nevertheless

Tutorials

IntroLM: Introspective Language Models via Prefilling-Time Self-Evaluation

arXiv:2601.03511v2 Announce Type: replace-cross Abstract: A major challenge for the operation of large language models (LLMs) is how to predict whether a specific LLM will produce sufficiently high-qu

tutorialsarxiv-cs-ai

Playing Games with My Heart: An Evaluation of AI Companion Apps

arXiv:2605.08093v1 Announce Type: cross Abstract: The use of chatbots for various forms of companionship is growing rapidly, raising a myriad of questions about simulated relationships, emotional depe

applicationsarxiv-cs-ai

Single-Configuration Attack Success Rate Is Not Enough: Jailbreak Evaluations Should Report Distributional Attack Success

arXiv:2605.09070v1 Announce Type: cross Abstract: Many jailbreak attack research papers report attack success rates for a limited number of parameter settings, even though there are many combinations

Evaluation of an Actuated Spine in Agile Quadruped Locomotion