Search: “evaluation”

arXiv:2605.05003v1 Announce Type: new Abstract: Reward models are a key component of large language model alignment, serving as proxies for human preferences during training. However, existing evaluat

Grok Imagine is an image quality model that has been discussed on the ComfyUI blog, likely covering its capabilities for evaluating or enhancing image generation quality within the ComfyUI framework.

arXiv:2605.03952v1 Announce Type: cross Abstract: Coding agents often pass per-prompt safety review yet ship exploitable code when their tasks are decomposed into routine engineering tickets. The chal

arXiv:2605.04352v1 Announce Type: new Abstract: We introduce a benchmark suite for evaluating structural mathematical reasoning in language models, built on subgroup-construction problems in SL(3, Z)

arXiv:2605.03101v1 Announce Type: new Abstract: Symbolic regression (SR), the task of discovering mathematical expressions that best describe a given dataset, remains a fundamental challenge in scient

arXiv:2605.04221v1 Announce Type: new Abstract: Clinical named entity recognition from dental progress notes is challenging because documentation is highly unstructured, domain-specific, and often pri

arXiv:2605.04461v1 Announce Type: new Abstract: While Test-Time Scaling (TTS) offers a promising direction to enhance video generation without the surging costs of training, current test-time video ge

arXiv:2605.03941v1 Announce Type: new Abstract: Achieving Artificial General Intelligence (AGI) requires agents that learn and interact adaptively, with interactive world models providing scalable env

arXiv:2605.01100v1 Announce Type: new Abstract: This work presents a knowledge-driven decision-support system that integrates structured defect knowledge with LLM-based reasoning to provide explainabl

arXiv:2605.03671v1 Announce Type: new Abstract: The most commonly used metrics for evaluating automatic speech transcriptions, namely Word Error Rate (WER) and Character Error Rate (CER), have been he

arXiv:2605.02661v1 Announce Type: new Abstract: Benchmarks within the OpenClaw ecosystem have thus far evaluated exclusively assistant-level tasks, leaving the academic-level capabilities of OpenClaw

arXiv:2605.03148v1 Announce Type: new Abstract: Reliable wildfire spread prediction is vital for risk-aware emergency planning, yet most deep learning models lack principled uncertainty quantification

arXiv:2605.03183v1 Announce Type: new Abstract: Evaluating canine electrocardiograms (ECGs) is challenging due to noise that can obscure clinically relevant cardiac electrical activity. Common sources

arXiv:2605.03984v1 Announce Type: new Abstract: Sampling from unnormalized densities is analogous to the generative modeling problem, but the target distribution is defined by a known energy function

arXiv:2510.12218v2 Announce Type: replace Abstract: Current approaches rely on zero-shot evaluation due to the absence of training data; while proprietary models such as GPT-4 exhibit strong reasoning

arXiv:2605.02962v1 Announce Type: new Abstract: Deep learning models for drug--target interaction (DTI) prediction often achieve strong benchmark performance without necessarily relying on mechanistic

arXiv:2605.01394v1 Announce Type: cross Abstract: Formal specification is essential for rigorous program verification, yet writing correct specifications remains costly and difficult to automate. Alth

arXiv:2602.00933v2 Announce Type: replace-cross Abstract: The Model Context Protocol (MCP) is rapidly becoming the standard interface for Large Language Models (LLMs) to discover and invoke external t

arXiv:2605.03482v1 Announce Type: cross Abstract: Persistent external memory enables LLM agents to maintain context across sessions, yet its security properties remain formally uncharacterized. We for

arXiv:2605.01566v1 Announce Type: new Abstract: Advances in inference methods have enabled language models to improve their predictions without additional training. These methods often prioritize raw

arXiv:2605.03129v1 Announce Type: cross Abstract: Browsing-enabled LLM assistants can fetch webpages and answer contact-seeking queries, creating a practical channel for scraping contact-style persona

AI coding agents are rapidly becoming ubiquitous across the software industry, fundamentally changing how developers write, test, and debug daily code. While these tools excel at localized, self-conta

arXiv:2605.03615v1 Announce Type: new Abstract: Engagement estimation from face video remains challenging because facial evidence is often incomplete, labeled data are limited, and engagement annotati

arXiv:2605.03919v1 Announce Type: cross Abstract: Reservoir geomodeling is central to subsurface characterization, but it remains challenging because conditioning data are sparse, geological heterogen

arXiv:2604.06573v2 Announce Type: replace Abstract: A Grammatical Error Correction (GEC) system produces a sequence of edits to correct an erroneous sentence. The quality of these edits is typically e

arXiv:2605.01359v1 Announce Type: new Abstract: In this paper, we employ the Minimal Cognitive Grid (MCG), a framework created to evaluate the cognitive plausibility of artificial systems, to offer a

The Structured Output Benchmark (SOB) dataset has been released and made available on Hugging Face. This dataset, hosted by Interfaze AI, likely provides benchmark data for evaluating models' ability

arXiv:2603.27343v2 Announce Type: replace Abstract: Existing large language models (LLMs) evaluations use fixed-difficulty benchmarks that cannot adapt as models improve, and rarely isolate specific c

arXiv:2605.01066v1 Announce Type: new Abstract: R2 score is the standard metric for evaluating regression tasks, offering a normalized magnitude-agnostic measure of accuracy that captures variance. Ho

arXiv:2605.02291v1 Announce Type: new Abstract: Video game engines have been an important source for generating large volumes of visual synthetic datasets for training and evaluating computer vision a

arXiv:2603.29895v2 Announce Type: replace-cross Abstract: We present a new theory of categorization based on an information-theoretic rational analysis. To evaluate this theory, we investigate how wel

arXiv:2510.18232v2 Announce Type: replace Abstract: Generating high-quality synthetic text under differential privacy (DP) is critical for training and evaluating language models without compromising

arXiv:2605.00462v1 Announce Type: cross Abstract: Intelligence Processing Units (IPU) have proven useful for many AI applications. In this paper, we evaluate them within the emerging field of AI for s

arXiv:2602.00074v2 Announce Type: replace-cross Abstract: While large language models (LLMs) can support clinical documentation needs, standalone tools struggle with 'workflow friction' from manual da

arXiv:2605.02660v1 Announce Type: cross Abstract: Predicting microsatellite instability (MSI) status from routine hematoxylin and eosin (H&E) whole slide images (WSIs) offers a practical alternative t

arXiv:2605.01020v1 Announce Type: new Abstract: This paper proposes and evaluates a new performance estimation method that leverages continual learning (CL) algorithms to carry out sequential simulati

arXiv:2605.01371v1 Announce Type: new Abstract: The rapid advancement of Multimodal Large Language Models (MLLMs) has empowered Unmanned Aerial Vehicle (UAV) with exceptional capabilities in spatial r

arXiv:2603.09625v2 Announce Type: replace Abstract: Deep learning models benefit from increasing data diversity and volume, motivating synthetic data augmentation to improve existing datasets. However

Ethan Mollick advocates for dedicated funding toward developing independent benchmarking systems for AI models, noting that evaluation is currently predominantly conducted by the labs that create the

arXiv:2605.00904v1 Announce Type: new Abstract: Learning-based fluence map prediction offers a fast alternative to iterative inverse planning in intensity-modulated radiation therapy (IMRT), but its r

arXiv:2605.00382v2 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly deployed to generate code for human-centered applications where demographic fairness is critical. Howeve

arXiv:2605.01452v1 Announce Type: cross Abstract: Existing evaluations of conformal prediction, such as prediction efficiency and test-conditional coverage, are defined in expectation over the calibra

arXiv:2601.08510v3 Announce Type: replace Abstract: Movie screenplays are rich long-form narratives that interleave complex character relationships, temporally ordered events, and dialogue-driven inte

arXiv:2605.02209v1 Announce Type: cross Abstract: Evaluating large language models across many benchmarks is expensive, yet many benchmarks are highly correlated. We formalize the selection of a small

arXiv:2504.20605v2 Announce Type: replace Abstract: Moral stories are a time-tested vehicle for transmitting values, yet modern NLP lacks a large, structured corpus that couples coherent narratives wi

arXiv:2605.02398v1 Announce Type: cross Abstract: As frontier AI models are deployed in high-stakes decision pipelines, their ability to maintain metacognitive stability -- knowing what they do not kn