Search: “evaluation”

A Large-Scale Study on the Accuracy vs Cost Trade-offs of Training and Evaluation Settings in Fine-Grained Image Recognition

arXiv:2605.18700v1 Announce Type: new Abstract: Prior work on fine-grained image recognition (FGIR) has established the importance of the backbone selection, but has neglected the accuracy-vs-cost tra

researcharxiv-cs-cv

19 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Systematic Evaluation of the Quality of Synthetic Clinical Notes Rephrased by LLMs at Million-Note Scale

DGX agent

arXiv:2605.17775v1 Announce Type: cross Abstract: Large language models (LLMs) can generate or synthesize clinical text for a wide range of applications, from improving clinical documentation to augme

researcharxiv-cs-ai

19 May 2026

Privacy Evaluation of Generative Models for Trajectory Generation

arXiv:2605.15246v1 Announce Type: new Abstract: Trajectory data is fundamental to modern urban intelligence, yet its sensitivity raises significant privacy concerns. Generative models such as Generati

researcharxiv-cs-lg

18 May 2026

CausalReasoningBenchmark: A Real-World Benchmark for Disentangled Evaluation of Causal Identification and Estimation

arXiv:2602.20571v2 Announce Type: replace Abstract: Many benchmarks for automated causal inference evaluate a system's performance based on a single numerical output, such as an Average Treatment Effe

15 May 2026

Logging Policy Design for Off-Policy Evaluation

arXiv:2605.15108v1 Announce Type: cross Abstract: Off-policy evaluation (OPE) estimates the value of a target treatment policy (e.g., a recommender system) using data collected by a different logging

15 May 2026

SCOOTER: A Human Evaluation Framework for Unrestricted Adversarial Examples

model-releasesarxiv-cs-cv

arXiv:2507.07776v3 Announce Type: replace Abstract: Unrestricted adversarial attacks aim to fool computer vision models without being constrained by ell_p-norm bounds to remain imperceptible to humans

15 May 2026

LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models

model-releasesarxiv-cs-cv

arXiv:2505.15616v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved significant advances in integrating visual and linguistic information, yet their ability to r

14 May 2026

CTFusion: A CTF-based Benchmark for LLM Agent Evaluation

model-releasesarxiv-cs-lg

arXiv:2605.11504v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) have enabled agentic systems for complex, multi-step tasks; cybersecurity is emerging as a prominent app

13 May 2026

ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox

arXiv:2605.10787v1 Announce Type: new Abstract: Current LLM agents are proficient at calling isolated APIs but struggle with the 'last mile' of commercial software automation. In real-world scenarios,

Applications

LLARS: Enabling Domain Expert & Developer Collaboration for LLM Prompting, Generation and Evaluation

arXiv:2605.10593v1 Announce Type: new Abstract: We demonstrate LLARS (LLM Assisted Research System), an open-source platform that bridges the gap between domain experts and developers for building LLM

applicationsarxiv-cs-ai

QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space

arXiv:2605.10458v1 Announce Type: new Abstract: Atomic properties such as partial charges or multipoles encode chemically meaningful information that can inform downstream molecular property predictio

researcharxiv-cs-lg

Threat Modelling using Domain-Adapted Language Models: Empirical Evaluation and Insights

arXiv:2605.10808v1 Announce Type: cross Abstract: Large Language Models(LLMs) are increasingly explored for cybersecurity applications such as vulnerability detection. In the domain of threat modellin

researcharxiv-cs-ai

Multi-Dimensional Evaluation of LLMs for Grammatical Error Correction

arXiv:2605.07635v1 Announce Type: new Abstract: Automated assistants for Grammatical Error Correction are now embedded in educational platforms serving millions of learners, yet three critical gaps re

Q-MMR: Off-Policy Evaluation via Recursive Reweighting and Moment Matching

arXiv:2605.06474v2 Announce Type: replace-cross Abstract: We present a novel theoretical framework, Q-MMR, for off-policy evaluation in finite-horizon MDPs. Q-MMR learns a set of scalar weights, one f

Robustness of Refugee-Matching Gains to Off-Policy Evaluation Choices

arXiv:2605.06686v1 Announce Type: new Abstract: Previous research has investigated the potential of refugee matching for boosting refugee outcomes, first considered by Bansak et al. (2018). This paper

safetyarxiv-cs-lg

Unsolvability Ceiling in Multi-LLM Routing: An Empirical Study of Evaluation Artifacts

arXiv:2605.07395v1 Announce Type: cross Abstract: Efficient routing across multiple LLMs enables cost-quality tradeoffs by directing queries to the cheapest capable model. Prior work attributes routin

VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents

arXiv:2506.21582v5 Announce Type: replace-cross Abstract: Text analytics has traditionally required specialized knowledge in Natural Language Processing (NLP) or text analysis, which presents a barrie

agentsarxiv-cs-ai

Hardware

Zero-Shot Neural Network Evaluation with Sample-Wise Activation Patterns

arXiv:2605.07378v1 Announce Type: new Abstract: Zero-shot proxies, also known as training-free metrics, are widely adopted to reduce the computational overhead in neural network evaluation for scenari

hardwarearxiv-cs-lg

Ensuring Reliability in Programming Knowledge Tracing: A Re-evaluation of Attention-augmented Models and Experimental Protocols

arXiv:2605.04727v1 Announce Type: new Abstract: Programming Knowledge Tracing (PKT) has recently advanced through hybrid approaches that integrate attention-based feature modeling for code representat

researcharxiv-cs-lg

7 May 2026

Unintended Negative Impacts of Promotional Language in Patent Evaluation

arXiv:2605.04926v1 Announce Type: new Abstract: Promotional language has been increasingly used to aid the communication of innovative ideas in science. Yet, less is known about its role in the contex

7 May 2026

Sim-to-Real Transfer and Robustness Evaluation of Reinforcement Learning Control with Integrated Perception on an ASV for Floating Waste Capture

arXiv:2605.02529v1 Announce Type: new Abstract: Autonomous surface vessels for floating-waste removal operate under varying hydrodynamics, external disturbances, and challenging water-surface percepti

agentsarxiv-cs-ro

5 May 2026

Can Small Language Models Handle Context-Summarized Multi-Turn Customer-Service QA? A Synthetic Data-Driven Comparative Evaluation

arXiv:2602.00665v3 Announce Type: replace Abstract: Customer-service question answering (QA) systems increasingly rely on conversational language understanding. While Large Language Models (LLMs) achi

safetyarxiv-cs-cl

4 May 2026

Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment

arXiv:2605.00022v1 Announce Type: new Abstract: The rapid proliferation of large audio models (LAMs) demands efficient approaches for model comparison, yet comprehensive benchmarks are costly. To fill

4 May 2026

Instruction Complexity Induces Positional Collapse in Adversarial LLM Evaluation

arXiv:2604.27249v1 Announce Type: cross Abstract: When instructed to underperform on multiple-choice evaluations, do language models engage with question content or fall back on positional shortcuts?

1 May 2026

Qualitative Evaluation of Language Model Rescoring in Automatic Speech Recognition

arXiv:2604.27533v1 Announce Type: new Abstract: Evaluating automatic speech recognition (ASR) systems is a classical but difficult and still open problem, which often boils down to focusing only on th

1 May 2026

From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents

model-releasesarxiv-cs-lg

arXiv:2603.25342v2 Announce Type: replace Abstract: Deep Research Agents (DRAs) aim to answer complex questions by searching the web, checking evidence, and synthesizing conclusions across heterogeneo

30 Apr 2026

Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models

arXiv:2512.20677v4 Announce Type: replace-cross Abstract: The increasing deployment of large language models (LLMs) in safety-critical applications raises fundamental challenges in systematically eval

safetyarxiv-cs-cl

29 Apr 2026

PSI-Bench: Towards Clinically Grounded and Interpretable Evaluation of Depression Patient Simulators

arXiv:2604.25840v1 Announce Type: new Abstract: Patient simulators are gaining traction in mental health training by providing scalable exposure to complex and sensitive patient interactions. Simulati

29 Apr 2026

VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation

arXiv:2604.25235v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly used as automated judges for multimodal systems, yet their scores provide no indication of reliability.

29 Apr 2026

An Empirical Evaluation of Locally Deployed LLMs for Bug Detection in Python Code

arXiv:2604.23361v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated strong performance on a wide range of software engineering tasks, including code generation and analysi

An empirical evaluation of the risks of AI model updates using clinical data: stability, arbitrariness, and fairness

arXiv:2604.23954v1 Announce Type: new Abstract: Artificial Intelligence and Machine Learning (AI/ML) models used in clinical settings are increasingly deployed to support clinical decision-making. How

Applications

Expert Evaluation of LLM's Open-Ended Legal Reasoning on the Japanese Bar Exam Writing Task

arXiv:2604.23730v1 Announce Type: new Abstract: Large language models (LLMs) have shown strong performance on legal benchmarks, including multiple-choice components of bar exams. However, their capaci

applicationsarxiv-cs-ai

Peer Identity Bias in Multi-Agent LLM Evaluation: An Empirical Study Using the TRUST Democratic Discourse Analysis Pipeline

arXiv:2604.22971v1 Announce Type: cross Abstract: The TRUST democratic discourse analysis pipeline exposes its large language model (LLM) components to peer model identity through multiple structural

Secure On-Premise Deployment of Open-Weights Large Language Models in Radiology: An Isolation-First Architecture with Prospective Pilot Evaluation

arXiv:2604.22768v1 Announce Type: cross Abstract: Purpose: To design, implement, evaluate, and report on the regulatory requirements of a self-hosted LLM infrastructure for radiology adhering to the p

StratRAG: A Multi-Hop Retrieval Evaluation Dataset for Retrieval-Augmented Generation Systems

arXiv:2604.22757v1 Announce Type: cross Abstract: We introduce StratRAG, an open-source retrieval evaluation dataset for benchmarking Retrieval-Augmented Generation (RAG) systems on multi-hop reasonin

Rethinking Cross-Domain Evaluation for Face Forgery Detection with Semantic Fine-grained Alignment and Mixture-of-Experts

arXiv:2604.21478v1 Announce Type: new Abstract: Nowadays, visual data forgery detection plays an increasingly important role in social and economic security with the rapid development of generative mo

safetyarxiv-cs-cv

24 Apr 2026

A Network-Aware Evaluation of Distributed Energy Resource Control in Smart Distribution Systems

arXiv:2604.19715v1 Announce Type: new Abstract: Distribution networks with high penetration of Distributed Energy Resources (DERs) increasingly rely on communication networks to coordinate grid-intera

researcharxiv-cs-cv

22 Apr 2026

When Graph Structure Becomes a Liability: A Critical Re-Evaluation of Graph Neural Networks for Bitcoin Fraud Detection under Temporal Distribution Shift

arXiv:2604.19514v1 Announce Type: cross Abstract: The consensus that GCN, GraphSAGE, GAT, and EvolveGCN outperform feature-only baselines on the Elliptic Bitcoin Dataset is widely cited but has not be

researcharxiv-cs-ai

22 Apr 2026

Beyond Word Boundaries: A Hebrew Coreference Benchmark and an Evaluation Protocol for Morphologically Complex Text

arXiv:2604.17108v1 Announce Type: new Abstract: Coreference Resolution (CR) is a fundamental NLP task critical for long-form tasks as information extraction, summarization, and many business applicati

21 Apr 2026

HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Models

arXiv:2511.01066v3 Announce Type: replace Abstract: We present an ongoing initiative to provide open, very large, high-quality, and richly annotated textual datasets for almost 200 languages. At 30 tr

21 Apr 2026

Spotlights and Blindspots: Evaluation Machine-Generated Text Detection

arXiv:2604.16607v1 Announce Type: new Abstract: With the rise of generative language models, machine-generated text detection has become a critical challenge. A wide variety of models is available, bu

21 Apr 2026

CTSCAN: Evaluation Leakage in Chest CT Segmentation and a Reproducible Patient-Disjoint Benchmark

model-releasesarxiv-cs-cv

arXiv:2604.15561v1 Announce Type: cross Abstract: Reported chest CT segmentation performance can be strongly inflated when train and test partitions mix slices from the same study. We present CTSCAN,

20 Apr 2026

Enabling Predictive Maintenance in District Heating Substations: A Labelled Dataset and Fault Detection Evaluation Framework based on Service Data

arXiv:2511.14791v2 Announce Type: replace-cross Abstract: Early detection of faults in district heating substations is imperative to reduce return temperatures and enhance efficiency. However, progres

20 Apr 2026

Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluation

arXiv:2601.07338v2 Announce Type: replace Abstract: Large Language Models (LLMs) have significantly advanced Machine Translation (MT), applying them to linguistically complex domains-such as Social Ne

agentsarxiv-cs-cl

17 Apr 2026

Blinded Multi-Rater Comparative Evaluation of a Large Language Model and Clinician-Authored Responses in CGM-Informed Diabetes Counseling

arXiv:2604.15124v1 Announce Type: new Abstract: Continuous glucose monitoring (CGM) is central to diabetes care, but explaining CGM patterns clearly and empathetically remains time-intensive. Evidence

safetyarxiv-cs-cl

17 Apr 2026

IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation

arXiv:2511.01014v3 Announce Type: replace Abstract: Instruction-following is a fundamental ability of Large Language Models (LLMs), requiring their generated outputs to follow multiple constraints imp

17 Apr 2026

A Dataset and Evaluation for Complex 4D Markerless Human Motion Capture

arXiv:2604.12765v1 Announce Type: new Abstract: Marker-based motion capture (MoCap) systems have long been the gold standard for accurate 4D human modeling, yet their reliance on specialized hardware

safetyarxiv-cs-cv

15 Apr 2026

Development, Evaluation, and Deployment of a Multi-Agent System for Thoracic Tumor Board