Search: “evaluation”

Bridging Reconstruction and Generation: A Latent Distribution Perspective on Evaluation and Improvement

arXiv:2609.24088v1 Announce Type: new Abstract: In latent generative models, reconstruction quality is often assumed to correlate with generative performance. However, reconstruction FID (rFID) can ex

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants

DGX agent

arXiv:2509.08494v2 Announce Type: replace-cross Abstract: As humans delegate more tasks and decisions to artificial intelligence (AI), we risk losing control of our individual and collective futures.

researcharxiv-cs-cl

22 Sept 2026

Initial Evaluation of Potential Bias in Coverage of Humans in Wikidata

arXiv:2609.22375v1 Announce Type: cross Abstract: Introduction. Open collaborative knowledge graphs such as Wikidata increasingly ground agentic artificial intelligence, information retrieval, and lan

Is Imagination Derived from Hallucination? A Cross-Taxonomy Evaluation of Imagination and Hallucination in Large Language Models

model-releasesarxiv-cs-cl

arXiv:2609.22152v1 Announce Type: new Abstract: Imagination performs as a high-level function of large language models (LLMs) which determines the potential of how an LLM creates unseen or creative co

Applications

Preserving What Matters: Semantic Scaffolds Beyond Saturation in Summarization Evaluation

arXiv:2609.22603v1 Announce Type: new Abstract: Summarization ships in countless production systems, making model selection a routine decision that depends on measuring summary quality. Existing metri

applicationsarxiv-cs-cl

CASCADE Against Jailbreaks: Combination Across Stages with Controlled Attack-Defense Evaluation

arXiv:2609.21793v1 Announce Type: cross Abstract: Defenses against jailbreak attacks on Large Language Models (LLMs) operate at different pipeline stages, such as input modification or output guard, b

21 Sept 2026

CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition

arXiv:2609.21259v1 Announce Type: new Abstract: Understanding and modeling human intelligence are parallel goals shared by artificial intelligence (AI) and cognitive science. As AI systems grow increa

researcharxiv-cs-ai

21 Sept 2026

Not All Nodes Are Created Equal: Homophily-Aware Stratification for Stable GNN Evaluation

arXiv:2609.19210v1 Announce Type: cross Abstract: Graph neural networks are widely used for transductive node classification, with accuracy typically measured on randomly drawn train/validation/test s

18 Sept 2026

VakQA: A Benchmark and Evaluation Study for Telugu Spoken Factoid Question Answering

model-releasesarxiv-cs-cl

arXiv:2609.19879v1 Announce Type: new Abstract: Question answering has advanced rapidly with large language models, but predominantly for high-resource languages, in both text and spoken settings. Spo

18 Sept 2026

Generic Characteristic-Zero Equivalence Between Derivative Bezout Inversion and Multipoint Evaluation

arXiv:2609.17578v1 Announce Type: cross Abstract: Let a_1,ldots,a_m be distinct elements of a field K, and let Z(X)=prod_{i=1}^m (X-a_i). We study the arithmetic complexity of computing the unique nor

researcharxiv-cs-lg

17 Sept 2026

Human-Centric Grasp State Assessment: Toward Transferring Subjective Evaluation to Robots

arXiv:2609.17540v1 Announce Type: new Abstract: We propose a framework that transfers tacit human subjective criteria to robotic systems for the appropriate grasping of deformable objects. Achieving s

researcharxiv-cs-ro

17 Sept 2026

EviSI: An Evidence-Based Evaluation Agent for Simultaneous Interpreting

arXiv:2609.08171v2 Announce Type: replace Abstract: Low-latency simultaneous speech-to-speech translation must keep pace with ongoing speech while preserving key information. To meet these demands, sy

agentsarxiv-cs-cl

16 Sept 2026

A Personalized Dynamic Balance Evaluation Paradigm for Hip Exoskeleton-Assisted Walking under Unexpected Ground Perturbations

arXiv:2609.14765v1 Announce Type: cross Abstract: Hip exoskeletons may improve recovery from unexpected gait perturbations, yet personalizing assistance remains difficult because balance is multidimen

researcharxiv-cs-lg

Automated Perceptually-Motivated Assessment of Photographic Consistency in Paired Clinical Photographs: Pipeline Development and Internal Evaluation

arXiv:2609.15144v1 Announce Type: new Abstract: Purpose: Paired pre- and post-operative photographs are the standard unit of evidence for plastic surgical outcomes, yet no objective metric verifies wh

Forward-Facing Near-Infrared Adds Little to Colour for Farm-Machinery Traversability: A Site-Disjoint Evaluation of Sensor-Dependent Spatial Leakage

arXiv:2609.13265v1 Announce Type: cross Abstract: Near-infrared (NIR) imaging does not consistently outperform standard color cameras for daytime agricultural traversability once spatial data leakage

agentsarxiv-cs-ai

Fusing Spectral Signatures and Activation Clustering for Backdoor Detection in Healthcare Imaging Models: Method, Implementation, and Evaluation

arXiv:2609.14290v1 Announce Type: cross Abstract: Machine learning models are increasingly deployed in healthcare imaging pipelines for diagnostic support, and training-time attacks against them are a

Evaluation of Vision-Language Models Across Diverse Coastal Environments

arXiv:2609.10855v1 Announce Type: new Abstract: Vision-language models (VLMs) enable robotic per- ception by associating visual observations with natural-language concepts. Yet their performance in co

safetyarxiv-cs-cv

11 Sept 2026

Adversarial Training for Tabular Credit Scoring: A Multi-Attack Robustness Evaluation in P2P Lending

arXiv:2609.09945v1 Announce Type: new Abstract: Machine learning-based credit scoring is increasingly central to Peer-to-Peer (P2P) lending, yet its resilience to adversarial manipulation, where appli

10 Sept 2026

Graph-Based Personalized Memory for LLM Agents: Representation, Evolution, Retrieval, and Evaluation

arXiv:2609.08599v1 Announce Type: new Abstract: Large Language Model (LLM) agents are evolving from single-session tools toward long-term personal assistants that must adapt to individual users across

agentsarxiv-cs-ai

10 Sept 2026

Rethinking the Evaluation of Efficiency Methods for Multi-Agent Systems

arXiv:2609.05933v1 Announce Type: new Abstract: Efficiency is increasingly important for Large Language Model (LLM)-based multi-agent systems (MAS), as larger models and more agents introduce substant

9 Sept 2026

An Evaluation Framework for Generating Multi-View Images of a Person in a Scene

arXiv:2609.04603v1 Announce Type: new Abstract: Recent generative image-editing Diffusion Transformers (DiTs) demonstrate impressive semantic editing capabilities but still struggle with spatially con

7 Sept 2026

Development and Evaluation of Ultrasound Image Learning Pipelines for MASLD Risk Stratification

arXiv:2609.04390v1 Announce Type: cross Abstract: Metabolic dysfunction-associated steatotic liver disease (MASLD) affects approximately 30% of the general population. Ultrasound-based imaging, includ

7 Sept 2026

Beyond Endpoint Scores: Time- and Capacity-Conditioned Evaluation of Continual Knowledge Updating

arXiv:2609.03900v1 Announce Type: new Abstract: Continual knowledge-updating methods are often declared superior from one final checkpoint and one conventional adapter rank. We show that this can be i

4 Sept 2026

Reducing Catastrophic Risk from AI with Systematic Monitoring and Evaluation of Rogue AI Progression

arXiv:2609.03189v1 Announce Type: cross Abstract: This article presents a structured framework of behavioral indicators that may signal progression toward potentially catastrophic threats from artific

researcharxiv-cs-ai

4 Sept 2026

Do Cantonese-Adapted Language Models Better Predict Cantonese Reading? A Cross-Model Eye-Tracking Evaluation

arXiv:2609.02163v1 Announce Type: new Abstract: Information-theoretic measures derived from autoregressive language models are widely used to characterize the expectations that shape human reading, bu

3 Sept 2026

Ten Architectures, One Error: Shared Failure Modes in Hyperspectral Classification under Spatially Disjoint Evaluation

model-releasesarxiv-cs-cv

arXiv:2609.01786v1 Announce Type: new Abstract: Hyperspectral image classification still relies heavily on random pixel splits within a single scene. The Salinas dataset, randomly split, is among the

3 Sept 2026

A Closed-Loop Evaluation of Capability Loss and Recovery in Compressed Driving Policies

arXiv:2609.00718v1 Announce Type: new Abstract: Many automobile and mobility companies deploy learned driving policies on embedded computers with limited memory and power. Pruning, knowledge distillat

safetyarxiv-cs-ai

2 Sept 2026

Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges

arXiv:2607.19011v2 Announce Type: replace-cross Abstract: Multimodal humor in memes, cartoons, and comics remains difficult for AI systems because intended meaning depends on non-literal mechanisms, s

2 Sept 2026

Synthetic Worlds for Temporal Evaluation and Knowledge Updating in LLMs

model-releasesarxiv-cs-cl

arXiv:2609.00184v1 Announce Type: new Abstract: Large language models (LLMs) rely on static pretraining corpora, causing their knowledge to become outdated over time. Existing approaches for evaluatin

2 Sept 2026

EDGE: Engine for Deterministic Graph Evaluation through Conversation Simulation from Graph Structured DSL Configuration

arXiv:2608.29971v1 Announce Type: new Abstract: As agentic systems evolve into complex multi agent orchestration workflows, there is a growing and critical need for systematic frameworks that measures

agentsarxiv-cs-ai

One Capability or Many? Testing the Economic Validity of Frontier AI Evaluation

arXiv:2608.29420v1 Announce Type: new Abstract: Frontier-model leaderboards now rank systems based on economic benchmarks, tests of how well models carry out professional tasks from software engineeri

Scalable Clinical Data Infrastructure and Comparative ML Evaluation for Hospitalisation Risk Prediction in Elderly Patients with Multiple Long-Term Conditions using CPRD

arXiv:2608.29419v1 Announce Type: cross Abstract: Deep learning architectures are increasingly proposed for patient trajectory modeling in electronic health records (EHRs), yet their advantage over si

Sketch2Inspire: Structure-Sensitive Evaluation for Product Retrieval

arXiv:2608.29364v1 Announce Type: new Abstract: Early-stage product design retrieval often requires more than category recognition: designers may need reference examples that match both a short semant

3D MRI-Based Alzheimer's Disease Classification Using Multi-Modal 3D CNN with Leakage-Aware Subject-Level Evaluation

model-releasesarxiv-cs-cv

arXiv:2603.17304v2 Announce Type: replace Abstract: Deep learning has become an important tool for Alzheimer's disease (AD) classification from structural MRI. Many existing studies analyze individual

31 Aug 2026

Mitigating Fabrication in Multi-Stage LLM Pipelines for Hiring: An Empirical Evaluation of Prompt Guardrails and Human-in-the-Loop Checkpoints

arXiv:2608.26171v1 Announce Type: cross Abstract: Multi-stage LLM hiring pipelines (resume improvement, interview question generation, answer feedback) can fabricate credentials, inflate qualifiers, a

28 Aug 2026

Local Ai

Deep Learning Segmentation of Diffusion-Weighted MRI Acute Ischaemic Stroke: A Pragmatic Evaluation Across Three Datasets

arXiv:2608.25675v1 Announce Type: new Abstract: Objective: Diffusion-weighted MRI (DWI-MRI) is the gold standard for visualizing and quantifying acute ischaemic stroke (AIS). Although deep learning me

local-aiarxiv-cs-cv

27 Aug 2026

Why ML-based cough models do not generalize: a systematic cross-dataset evaluation for tuberculosis screening

arXiv:2608.25846v1 Announce Type: cross Abstract: Cough acoustics are promising for non-invasive tuberculosis (TB) screening, yet whether machine learning (ML) models capture disease-related acoustics

researcharxiv-cs-lg

27 Aug 2026

From Empirical Evaluation to Context-Aware Enhancement: Repairing Regression Errors with LLMs

arXiv:2506.13182v3 Announce Type: replace-cross Abstract: [...] Since then, various APR approaches, especially those leveraging the power of large language models (LLMs), have been rapidly developed t

26 Aug 2026

ChemDIRT: A Diversified Instruction, Representation, and Task Benchmark for Robust Chemistry-LLM Evaluation

arXiv:2608.21504v1 Announce Type: new Abstract: The rapid advancement of large language models (LLMs) has led to increasing interest in their application to scientific domains such as chemistry. Howev

Development and Feasibility Evaluation of an Edge AI as Medical Device System for Breast Cancer Multidisciplinary Team Meetings

arXiv:2608.22108v1 Announce Type: new Abstract: Breast Cancer Multidisciplinary Team (MDT) meetings manage increasingly complex cases under considerable time pressure, and documentation requirements c

Applications

Diversity-Based Active Learning: An Evaluation of Metric Spaces for Active Learning Selection

arXiv:2608.23461v1 Announce Type: new Abstract: With rapid advancement over the last few years, many different methods are now widely used for classification. However, training these models requires s

applicationsarxiv-cs-lg

Evaluation of Small Vision-Language Models on Qualitative Mechanical Problems

arXiv:2608.22143v1 Announce Type: new Abstract: Qualitative mechanical problem-solving (QMPS) refers to solving qualitative problems from the mechanical domain. Qualitative problems can be solved with

Local Ai

Operational digital twin clinics enable task-based evaluation of embodied AI

arXiv:2608.21416v1 Announce Type: cross Abstract: Embodied artificial intelligence (AI) must be tested in the clinical environments where it will operate, but building realistic, robot-testable settin

local-aiarxiv-cs-ai

From Regulation to Implementation: A Critical Evaluation of LLM-Assisted Regulatory Compliance in Industry

arXiv:2608.21317v1 Announce Type: new Abstract: The European Union (EU) has emerged as a leading regulatory body in the development of sustainability and privacy regulations. While new regulation requ

researcharxiv-cs-ai

24 Aug 2026

Applications

Machine Learning and ARIMA Model Averaging for Adaptive Public Health Forecasting: Comparative Evaluation and an Ontario COVID-19 Case Study

arXiv:2608.20406v1 Announce Type: new Abstract: Public health forecasts must respond to abrupt changes in surveillance data without over-extrapolating noise, reporting artifacts, or temporary trends.

applicationsarxiv-cs-lg

24 Aug 2026

A Speech Corpus for Mizo Automatic Speech Recognition: Whisper and SraVaani 1.0 Fine-Tuning with Morphology-Aware Evaluation

arXiv:2608.19361v1 Announce Type: new Abstract: This study reports the development of an Automatic Speech Recognition (ASR) system in Mizo, a low-resource language. The development included collecting

researcharxiv-cs-cl

21 Aug 2026

Building real-time digital twin instances with Function+Data Flow: user evaluation and extension for iterative pipelines

arXiv:2608.18480v1 Announce Type: cross Abstract: Digital twins (DTs) increasingly leverage artificial intelligence (AI) and machine learning (ML) pipelines, both to build real-time DTs from high-fide

researcharxiv-cs-cl

20 Aug 2026

Communicating Credit Risk with Large Language Models: Evaluation of Explanations from Standard and Alternative Data-Based Models

arXiv:2608.17715v1 Announce Type: cross Abstract: Credit decisioning is a high-stakes task in which model outputs must be accurate and explainable to support compliant decisions. Although modern credi