Search: “evaluation”

arXiv:2604.06403v1 Announce Type: cross Abstract: The paper presents an approach for the recognition of toxic habits named entities in Spanish clinical texts. The approach was developed for the ToxHab

arXiv:2603.15118v2 Announce Type: replace Abstract: We introduce VAREX (VARied-schema EXtraction), a benchmark for evaluating multimodal foundation models on structured data extraction from government

arXiv:2610.17702v1 Announce Type: new Abstract: Evaluating the quality of process models discovered from event logs is a fundamental challenge in process mining. Traditional evaluation relies on confo

arXiv:2610.17827v1 Announce Type: new Abstract: Existing approaches to training and evaluating search agents focus heavily on final-answer quality, largely neglecting how generated answers map back to

arXiv:2610.20138v1 Announce Type: new Abstract: Batch black-box optimization is important for expensive systems where many candidates can be evaluated in parallel, yet only a few rounds of evaluation

arXiv:2608.28281v2 Announce Type: replace Abstract: Loop Engineering is emerging as a practice for organizing development work around coding agents. Instead of writing each prompt by hand, practitione

arXiv:2610.19823v1 Announce Type: new Abstract: Text-to-video (T2V) generation models can produce visually realistic videos that nevertheless violate the physical consequences implied by the prompt an

arXiv:2610.19011v1 Announce Type: new Abstract: The Practical Molecular Optimization (PMO) benchmark standardized evaluation in molecular optimization, but it is built on simple property-based oracles

arXiv:2601.08689v3 Announce Type: replace Abstract: Large Language Models (LLMs) have shown strong capabilities in finance-oriented question answering, yet their evaluation on professional quantitativ

arXiv:2610.14720v1 Announce Type: new Abstract: World Action Models (WAMs) combine action generation with future prediction, allowing predicted futures to support action evaluation and robot decision

arXiv:2610.15452v1 Announce Type: new Abstract: Text-to-video (T2V) models are increasingly used to generate videos with targeted emotional expressions, yet existing evaluation methods mainly rely on

arXiv:2610.18826v1 Announce Type: cross Abstract: We introduce WorldOracle, a benchmark specifically designed to evaluate action-conditioned world models (ACWMs) against paired real-world executions.

arXiv:2610.12086v1 Announce Type: new Abstract: LLM-based program evolution relies on evaluation feedback to guide the iterative search for high-performing programs. However, evaluation is often compu

arXiv:2601.14994v2 Announce Type: replace-cross Abstract: Data contamination can invalidate benchmark evaluation when a model benefits from memorized evaluation content rather than genuine generalizat

arXiv:2610.02270v1 Announce Type: cross Abstract: Medical vision-language model (VLM) evaluation is sensitive to workflow design, prompting strategy, and benchmark construction, yet most studies treat

arXiv:2610.01162v1 Announce Type: new Abstract: Reliable video world models could provide scalable predictive environments for robot learning, planning, and evaluation. However, generated robot videos

arXiv:2609.39564v1 Announce Type: new Abstract: Delegating complete application development to coding agents requires preserving the intended design rather than simply producing plausible outputs thro

arXiv:2609.30756v1 Announce Type: new Abstract: Generative image communication transmits compact semantic tokens under a limited packet budget, where token selection directly affects the final reconst

arXiv:2609.24516v1 Announce Type: new Abstract: In recent years, large language models (LLMs) have emerged as a popular alternative for evaluation. Often referred to as LLMs as judges (LLJs), these sy

arXiv:2602.16313v2 Announce Type: replace Abstract: Existing evaluations of agents with memory typically assess memorization and action in isolation. One class of benchmarks evaluates memorization by

arXiv:2609.08765v1 Announce Type: cross Abstract: Large language model (LLM) benchmarks are often treated as fixed datasets with stable scores, yet their outcomes depend on configurable evaluation pip

arXiv:2609.07095v1 Announce Type: new Abstract: LLM assistants often produce more answers than humans can review before users see them. Most evaluations ask whether an answer is wrong, unsupported, or

arXiv:2609.07987v1 Announce Type: new Abstract: LLM-based digital twins promise to reduce repeated human data collection by generating person- specific responses, yet existing evaluations provide litt

arXiv:2511.22341v2 Announce Type: replace Abstract: Previous works identify sensitivity to option order as a key issue in multiple-choice VQA (MC-VQA) evaluation and propose protocols to mitigate this

arXiv:2609.01552v1 Announce Type: new Abstract: Scientific equation discovery has long been central to scientific progress, proceeding through iterative cycles of hypothesis generation, observational

arXiv:2608.28694v1 Announce Type: new Abstract: Long-horizon video generation is evaluated with whole-frame metrics that reward motion and temporal consistency. For fixed-camera nature scenes this cre

arXiv:2608.27477v1 Announce Type: cross Abstract: Graphical user interfaces have emerged as an important environment for evaluating autonomous AI agents on multimodal interactive tasks. Existing bench

arXiv:2608.26133v1 Announce Type: new Abstract: Evaluating AI agents that use external tools requires realistic test scenarios that capture how practitioners compose tools and iterate across conversat

arXiv:2608.26592v1 Announce Type: new Abstract: Following clinical decision pathways (CDPs) defined by clinical practice guidelines is essential for safe and reliable medical decision-making. However,

arXiv:2608.27391v1 Announce Type: new Abstract: LLMs are increasingly able to answer complex questions about enterprise-scale document collections. But evaluation is hard: companies don't want to shar

arXiv:2608.27340v1 Announce Type: new Abstract: Steering interventions targeting eval-awareness, a model's recognition that it is being tested, are increasingly used in safety evaluation pipelines, wh

arXiv:2608.25449v1 Announce Type: new Abstract: Formal theorem proving enables machine-verifiable evaluation of mathematical reasoning, yet existing benchmarks often emphasize aggregate proof accuracy

arXiv:2608.24551v1 Announce Type: cross Abstract: Machine learning models are widely used in financial fraud and credit-risk detection, yet their adversarial robustness remains difficult to evaluate b

arXiv:2608.24486v1 Announce Type: cross Abstract: Purpose: To quantify how evaluation annotations influence measured pulmonary embolism (PE) segmentation performance relative to model training changes

arXiv:2506.10903v2 Announce Type: replace Abstract: Statement autoformalization plays a crucial role in formal mathematical reasoning by enabling the automatic translation of natural language statemen

arXiv:2608.18888v1 Announce Type: new Abstract: The rapid advancement of Natural Language Generation (NLG) has made the reliable evaluation of generated text increasingly critical, as these systems, s

arXiv:2608.18091v1 Announce Type: cross Abstract: As LLM-as-a-judge systems become increasingly widespread, self-preference in LLMs -- the tendency to favor one's own outputs -- raises growing concern

arXiv:2608.12894v1 Announce Type: new Abstract: Cultural evaluation of large language models (LLMs) often focuses on high-resource standard languages, leaving regional culture and dialect communities

arXiv:2608.10268v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly mediate legal determinations over what human rights are realized, and how. Yet, no evaluation benchmark exis

arXiv:2608.03259v1 Announce Type: cross Abstract: As time-series foundation models have emerged, the need for benchmarks that can evaluate their forecasting ability in meaningful ways has become incre

arXiv:2607.28771v1 Announce Type: new Abstract: Medical foundation models (FMs) are increasingly used for brain MRI analysis. However, their evaluation remains dominated by high-resource datasets, lea

arXiv:2607.18529v2 Announce Type: replace-cross Abstract: Teaching videos are becoming a major medium for education, creating a growing need for scalable evaluation of their pedagogical quality. Exist

arXiv:2607.28802v1 Announce Type: new Abstract: Existing evaluations often reduce agent failures to system-level outcomes, obscuring where the fault originated and which intervention would improve the

arXiv:2607.28644v1 Announce Type: cross Abstract: Creativity in computational systems is often evaluated as an objective property of artifacts, with existing Computational Creativity (CC) frameworks a

arXiv:2607.24649v1 Announce Type: new Abstract: Large language models are increasingly used as social simulators, including as synthetic survey respondents. Most evaluations ask whether simulated outc

arXiv:2607.20270v1 Announce Type: new Abstract: Large language models are increasingly evaluated through the values they endorse, but such evaluations presuppose that models can identify the value exp