Search: “evaluation”

UltraBench 2: Towards Robust Evaluation of Vision Foundation Models on Ultrasound

model-releasesarxiv-cs-cv

arXiv:2609.28610v1 Announce Type: new Abstract: Benchmarking is an increasingly critical part of research in machine learning and the domains where it is applied, including healthcare. Yet, despite th

25 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

A Systematic Evaluation of Infrastructure-Based Radar System for Highway Traffic Monitoring

DGX agent

arXiv:2609.27143v1 Announce Type: new Abstract: Infrastructure-based radar systems offer robust and long-range solutions for traffic monitoring, yet their detection and tracking performance under real

model-releasesarxiv-cs-cv

24 Sept 2026

Multi-View Fusion for Encrypted C2 Detection: A Leakage-Controlled Measurement Study of Evaluation Pitfalls

arXiv:2609.27311v1 Announce Type: cross Abstract: Command-and-control (C2) traffic increasingly hides within TLS, so defenders now apply machine learning to traffic metadata. Many studies assume that

researcharxiv-cs-ai

24 Sept 2026

When Post-Processing Fairness Constraints Help and When They Harm: Evidence from Eight Cross-Domain Evaluations

arXiv:2609.26955v1 Announce Type: new Abstract: Fairness audits in production ML typically occur once, at deployment, on a single domain. Both fail in practice: fairness can shift after retraining or

24 Sept 2026

CraftBench-UE: Deterministic Evaluation for Coding Agents in Unreal Engine

arXiv:2609.23142v1 Announce Type: new Abstract: Building gameplay features in a game engine requires more than code, as code that compiles and runs does not necessarily implement the requested gamepla

ReasonLab: A Controlled and Auditable Evaluation of Prompting Techniques for Multiple-Choice QA

arXiv:2607.14109v2 Announce Type: replace Abstract: Probing the capabilities of Large Language Models (LLMs) and building robust solutions for Multiple-Choice Question Answering (MCQA) remain central

researcharxiv-cs-cl

VLAQuantBench: Closed-Loop Evaluation of Post-Training Quantization for Vision-Language-Action Models

model-releasesarxiv-cs-ro

arXiv:2609.25376v1 Announce Type: new Abstract: Post-training quantization reduces the memory requirements of vision-language-action (VLA) models, but precision selection must account for the interact

Zephyron: Integrated Design and Analytical Evaluation of a Solar-Assisted Mobile Manipulator for Multimodal Environmental Reconnaissance and Distributed Visual Inference

arXiv:2609.25709v1 Announce Type: new Abstract: Environmental reconnaissance needs mobile platforms that carry sensors, preserve measurement context, and return interpretable evidence under limited en

safetyarxiv-cs-ro

Bridging Reconstruction and Generation: A Latent Distribution Perspective on Evaluation and Improvement

arXiv:2609.24088v1 Announce Type: new Abstract: In latent generative models, reconstruction quality is often assumed to correlate with generative performance. However, reconstruction FID (rFID) can ex

HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants

arXiv:2509.08494v2 Announce Type: replace-cross Abstract: As humans delegate more tasks and decisions to artificial intelligence (AI), we risk losing control of our individual and collective futures.

researcharxiv-cs-cl

Initial Evaluation of Potential Bias in Coverage of Humans in Wikidata

arXiv:2609.22375v1 Announce Type: cross Abstract: Introduction. Open collaborative knowledge graphs such as Wikidata increasingly ground agentic artificial intelligence, information retrieval, and lan

Is Imagination Derived from Hallucination? A Cross-Taxonomy Evaluation of Imagination and Hallucination in Large Language Models

model-releasesarxiv-cs-cl

arXiv:2609.22152v1 Announce Type: new Abstract: Imagination performs as a high-level function of large language models (LLMs) which determines the potential of how an LLM creates unseen or creative co

Applications

Preserving What Matters: Semantic Scaffolds Beyond Saturation in Summarization Evaluation

arXiv:2609.22603v1 Announce Type: new Abstract: Summarization ships in countless production systems, making model selection a routine decision that depends on measuring summary quality. Existing metri

applicationsarxiv-cs-cl

CASCADE Against Jailbreaks: Combination Across Stages with Controlled Attack-Defense Evaluation

arXiv:2609.21793v1 Announce Type: cross Abstract: Defenses against jailbreak attacks on Large Language Models (LLMs) operate at different pipeline stages, such as input modification or output guard, b

21 Sept 2026

CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition

arXiv:2609.21259v1 Announce Type: new Abstract: Understanding and modeling human intelligence are parallel goals shared by artificial intelligence (AI) and cognitive science. As AI systems grow increa

researcharxiv-cs-ai

21 Sept 2026

Not All Nodes Are Created Equal: Homophily-Aware Stratification for Stable GNN Evaluation

arXiv:2609.19210v1 Announce Type: cross Abstract: Graph neural networks are widely used for transductive node classification, with accuracy typically measured on randomly drawn train/validation/test s

18 Sept 2026

VakQA: A Benchmark and Evaluation Study for Telugu Spoken Factoid Question Answering

model-releasesarxiv-cs-cl

arXiv:2609.19879v1 Announce Type: new Abstract: Question answering has advanced rapidly with large language models, but predominantly for high-resource languages, in both text and spoken settings. Spo

18 Sept 2026

Human-Centric Grasp State Assessment: Toward Transferring Subjective Evaluation to Robots

arXiv:2609.17540v1 Announce Type: new Abstract: We propose a framework that transfers tacit human subjective criteria to robotic systems for the appropriate grasping of deformable objects. Achieving s

researcharxiv-cs-ro

17 Sept 2026

EviSI: An Evidence-Based Evaluation Agent for Simultaneous Interpreting

arXiv:2609.08171v2 Announce Type: replace Abstract: Low-latency simultaneous speech-to-speech translation must keep pace with ongoing speech while preserving key information. To meet these demands, sy

agentsarxiv-cs-cl

16 Sept 2026

A Personalized Dynamic Balance Evaluation Paradigm for Hip Exoskeleton-Assisted Walking under Unexpected Ground Perturbations

arXiv:2609.14765v1 Announce Type: cross Abstract: Hip exoskeletons may improve recovery from unexpected gait perturbations, yet personalizing assistance remains difficult because balance is multidimen

researcharxiv-cs-lg

Automated Perceptually-Motivated Assessment of Photographic Consistency in Paired Clinical Photographs: Pipeline Development and Internal Evaluation

arXiv:2609.15144v1 Announce Type: new Abstract: Purpose: Paired pre- and post-operative photographs are the standard unit of evidence for plastic surgical outcomes, yet no objective metric verifies wh

Forward-Facing Near-Infrared Adds Little to Colour for Farm-Machinery Traversability: A Site-Disjoint Evaluation of Sensor-Dependent Spatial Leakage

arXiv:2609.13265v1 Announce Type: cross Abstract: Near-infrared (NIR) imaging does not consistently outperform standard color cameras for daytime agricultural traversability once spatial data leakage

agentsarxiv-cs-ai

Fusing Spectral Signatures and Activation Clustering for Backdoor Detection in Healthcare Imaging Models: Method, Implementation, and Evaluation

arXiv:2609.14290v1 Announce Type: cross Abstract: Machine learning models are increasingly deployed in healthcare imaging pipelines for diagnostic support, and training-time attacks against them are a

Evaluation of Vision-Language Models Across Diverse Coastal Environments

arXiv:2609.10855v1 Announce Type: new Abstract: Vision-language models (VLMs) enable robotic per- ception by associating visual observations with natural-language concepts. Yet their performance in co

safetyarxiv-cs-cv

11 Sept 2026

Adversarial Training for Tabular Credit Scoring: A Multi-Attack Robustness Evaluation in P2P Lending

arXiv:2609.09945v1 Announce Type: new Abstract: Machine learning-based credit scoring is increasingly central to Peer-to-Peer (P2P) lending, yet its resilience to adversarial manipulation, where appli

10 Sept 2026

Graph-Based Personalized Memory for LLM Agents: Representation, Evolution, Retrieval, and Evaluation

arXiv:2609.08599v1 Announce Type: new Abstract: Large Language Model (LLM) agents are evolving from single-session tools toward long-term personal assistants that must adapt to individual users across

agentsarxiv-cs-ai

10 Sept 2026

Rethinking the Evaluation of Efficiency Methods for Multi-Agent Systems

arXiv:2609.05933v1 Announce Type: new Abstract: Efficiency is increasingly important for Large Language Model (LLM)-based multi-agent systems (MAS), as larger models and more agents introduce substant

9 Sept 2026

An Evaluation Framework for Generating Multi-View Images of a Person in a Scene

arXiv:2609.04603v1 Announce Type: new Abstract: Recent generative image-editing Diffusion Transformers (DiTs) demonstrate impressive semantic editing capabilities but still struggle with spatially con

7 Sept 2026

Development and Evaluation of Ultrasound Image Learning Pipelines for MASLD Risk Stratification

arXiv:2609.04390v1 Announce Type: cross Abstract: Metabolic dysfunction-associated steatotic liver disease (MASLD) affects approximately 30% of the general population. Ultrasound-based imaging, includ

7 Sept 2026

Beyond Endpoint Scores: Time- and Capacity-Conditioned Evaluation of Continual Knowledge Updating

arXiv:2609.03900v1 Announce Type: new Abstract: Continual knowledge-updating methods are often declared superior from one final checkpoint and one conventional adapter rank. We show that this can be i

4 Sept 2026

Reducing Catastrophic Risk from AI with Systematic Monitoring and Evaluation of Rogue AI Progression

arXiv:2609.03189v1 Announce Type: cross Abstract: This article presents a structured framework of behavioral indicators that may signal progression toward potentially catastrophic threats from artific

researcharxiv-cs-ai

4 Sept 2026

Do Cantonese-Adapted Language Models Better Predict Cantonese Reading? A Cross-Model Eye-Tracking Evaluation

arXiv:2609.02163v1 Announce Type: new Abstract: Information-theoretic measures derived from autoregressive language models are widely used to characterize the expectations that shape human reading, bu

3 Sept 2026

Ten Architectures, One Error: Shared Failure Modes in Hyperspectral Classification under Spatially Disjoint Evaluation

model-releasesarxiv-cs-cv

arXiv:2609.01786v1 Announce Type: new Abstract: Hyperspectral image classification still relies heavily on random pixel splits within a single scene. The Salinas dataset, randomly split, is among the

3 Sept 2026

A Closed-Loop Evaluation of Capability Loss and Recovery in Compressed Driving Policies

arXiv:2609.00718v1 Announce Type: new Abstract: Many automobile and mobility companies deploy learned driving policies on embedded computers with limited memory and power. Pruning, knowledge distillat

safetyarxiv-cs-ai

2 Sept 2026

Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges

arXiv:2607.19011v2 Announce Type: replace-cross Abstract: Multimodal humor in memes, cartoons, and comics remains difficult for AI systems because intended meaning depends on non-literal mechanisms, s

2 Sept 2026

Synthetic Worlds for Temporal Evaluation and Knowledge Updating in LLMs

model-releasesarxiv-cs-cl

arXiv:2609.00184v1 Announce Type: new Abstract: Large language models (LLMs) rely on static pretraining corpora, causing their knowledge to become outdated over time. Existing approaches for evaluatin

2 Sept 2026

EDGE: Engine for Deterministic Graph Evaluation through Conversation Simulation from Graph Structured DSL Configuration

arXiv:2608.29971v1 Announce Type: new Abstract: As agentic systems evolve into complex multi agent orchestration workflows, there is a growing and critical need for systematic frameworks that measures

agentsarxiv-cs-ai

One Capability or Many? Testing the Economic Validity of Frontier AI Evaluation

arXiv:2608.29420v1 Announce Type: new Abstract: Frontier-model leaderboards now rank systems based on economic benchmarks, tests of how well models carry out professional tasks from software engineeri

Scalable Clinical Data Infrastructure and Comparative ML Evaluation for Hospitalisation Risk Prediction in Elderly Patients with Multiple Long-Term Conditions using CPRD

arXiv:2608.29419v1 Announce Type: cross Abstract: Deep learning architectures are increasingly proposed for patient trajectory modeling in electronic health records (EHRs), yet their advantage over si

Sketch2Inspire: Structure-Sensitive Evaluation for Product Retrieval

arXiv:2608.29364v1 Announce Type: new Abstract: Early-stage product design retrieval often requires more than category recognition: designers may need reference examples that match both a short semant

3D MRI-Based Alzheimer's Disease Classification Using Multi-Modal 3D CNN with Leakage-Aware Subject-Level Evaluation

model-releasesarxiv-cs-cv

arXiv:2603.17304v2 Announce Type: replace Abstract: Deep learning has become an important tool for Alzheimer's disease (AD) classification from structural MRI. Many existing studies analyze individual

31 Aug 2026

Mitigating Fabrication in Multi-Stage LLM Pipelines for Hiring: An Empirical Evaluation of Prompt Guardrails and Human-in-the-Loop Checkpoints

arXiv:2608.26171v1 Announce Type: cross Abstract: Multi-stage LLM hiring pipelines (resume improvement, interview question generation, answer feedback) can fabricate credentials, inflate qualifiers, a

28 Aug 2026

Local Ai

Deep Learning Segmentation of Diffusion-Weighted MRI Acute Ischaemic Stroke: A Pragmatic Evaluation Across Three Datasets

arXiv:2608.25675v1 Announce Type: new Abstract: Objective: Diffusion-weighted MRI (DWI-MRI) is the gold standard for visualizing and quantifying acute ischaemic stroke (AIS). Although deep learning me

local-aiarxiv-cs-cv

27 Aug 2026

Why ML-based cough models do not generalize: a systematic cross-dataset evaluation for tuberculosis screening

arXiv:2608.25846v1 Announce Type: cross Abstract: Cough acoustics are promising for non-invasive tuberculosis (TB) screening, yet whether machine learning (ML) models capture disease-related acoustics

researcharxiv-cs-lg

27 Aug 2026

From Empirical Evaluation to Context-Aware Enhancement: Repairing Regression Errors with LLMs

arXiv:2506.13182v3 Announce Type: replace-cross Abstract: [...] Since then, various APR approaches, especially those leveraging the power of large language models (LLMs), have been rapidly developed t

26 Aug 2026

ChemDIRT: A Diversified Instruction, Representation, and Task Benchmark for Robust Chemistry-LLM Evaluation

arXiv:2608.21504v1 Announce Type: new Abstract: The rapid advancement of large language models (LLMs) has led to increasing interest in their application to scientific domains such as chemistry. Howev

25 Aug 2026

Development and Feasibility Evaluation of an Edge AI as Medical Device System for Breast Cancer Multidisciplinary Team Meetings

arXiv:2608.22108v1 Announce Type: new Abstract: Breast Cancer Multidisciplinary Team (MDT) meetings manage increasingly complex cases under considerable time pressure, and documentation requirements c

25 Aug 2026

Applications

Diversity-Based Active Learning: An Evaluation of Metric Spaces for Active Learning Selection