Search: “evaluation”

arXiv:2510.14980v3 Announce Type: replace Abstract: Large language models (LLMs) have shown strong abilities in writing and revising programs, yet many program-synthesis benchmarks still evaluate prog

arXiv:2609.01453v1 Announce Type: cross Abstract: Dexterous manipulation policies learned by imitation are typically evaluated for robustness to variation in scenes, objects, or instructions, but thei

arXiv:2609.01249v1 Announce Type: cross Abstract: Invisible watermarks are typically evaluated against predefined perturbations such as compression, blur, noise, cropping, and denoising. Public founda

arXiv:2609.01556v1 Announce Type: cross Abstract: We evaluate embedding retrieval where surface form and meaning are pulled apart on purpose: retrieving items that share underlying structure but not w

arXiv:2608.29461v1 Announce Type: new Abstract: Sandbagging models strategically underperform on evaluations while retaining the capabilities being measured. The evaluations that guide frontier-model

arXiv:2608.31111v1 Announce Type: new Abstract: Many important forms of human learning begin with a vague goal, such as 'become a better physicist' or 'improve at research.' Learners must interpret th

arXiv:2607.10286v2 Announce Type: replace Abstract: Large language model (LLM) agents are increasingly used in trading systems, where model reasoning, tool use, and continual decisions incur costs tha

arXiv:2601.18486v3 Announce Type: replace Abstract: Demographic cue-based evaluation is widely used to study how large language models (LLMs) adapt their responses to signaled demographic attributes w

arXiv:2608.28626v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to generate peer reviews, prompting examination of their capacity for critical evaluation. This stu

arXiv:2608.28591v1 Announce Type: new Abstract: Recent advancements in AI are helping scientists achieve breakthroughs in fields such as mathematics, medicine, and materials sciences. New evaluation d

arXiv:2608.30653v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in multi-image scenarios requiring complex reasoning across visual contexts. Howeve

arXiv:2601.08626v4 Announce Type: replace Abstract: Large language models (LLMs) excel at semantic understanding, yet their ability to reconstruct internal structure from scrambled inputs remains unde

arXiv:2608.28762v1 Announce Type: new Abstract: Recent advances in visual question answering (VQA) and multimodal large language models (MLLMs) have enabled natural-language reasoning over traffic sce

arXiv:2608.29257v1 Announce Type: new Abstract: Multiple-choice questions (MCQs) are a standard format for evaluating large language models (LLMs), yet the popularity of answer options can confound ev

arXiv:2608.30748v1 Announce Type: cross Abstract: Existing jailbreak evaluations typically characterize robustness using a single attack success rate (ASR) measured in a default configuration (the van

arXiv:2608.28610v1 Announce Type: new Abstract: Existing LLM moral evaluations typically present models with isolated moral vignettes and elicit a single-shot decision, neglecting a factor known to pr

arXiv:2608.29608v1 Announce Type: cross Abstract: Machine learning is usually evaluated after an evidence interface has been fixed. A dataset, sensor suite, query language, action set, or experimental

arXiv:2509.22737v3 Announce Type: replace Abstract: Visual comparison reasoning is a fundamental capability of vision-language models (VLMs), covering judgments of object quantity, geometric dimension

arXiv:2511.14684v2 Announce Type: replace Abstract: Large language models (LLMs) often make reasoning errors when solving mathematical problems, and how to automatically detect and correct these error

arXiv:2608.26186v1 Announce Type: cross Abstract: This study examines how prompt and response language influence the behavior of large language models. Using five models, we evaluated answers to 68 no

arXiv:2608.26295v1 Announce Type: cross Abstract: Tool-augmented LLMs must arbitrate between two fallible sources when a tool return conflicts with their parametric memory, yet existing evaluations me

arXiv:2608.26317v1 Announce Type: cross Abstract: Frontier language models are increasingly marketed as omni systems that can perceive and respond across modalities. Existing evaluation frameworks, ho

arXiv:2608.27004v1 Announce Type: new Abstract: Reliable evaluation of vision-language models (VLMs) and medical vision-language models (Medical-VLMs) requires calibrated confidence, particularly unde

arXiv:2608.25823v1 Announce Type: new Abstract: Accurate regional near-surface temperature forecasting is fundamental to short-range weather services and downstream risk assessment. Existing deep lear

arXiv:2608.26036v1 Announce Type: cross Abstract: Answer accuracy is an insufficient reliability signal for LLM data agents. In structured-data tasks, a benchmark-correct answer can be produced by an

arXiv:2608.24885v1 Announce Type: cross Abstract: Action-conditioned world models are increasingly used as learned simulators for policy evaluation and improvement, yet their effectiveness rests on an

arXiv:2608.21839v1 Announce Type: new Abstract: Reliable reward models are essential for text-to-video evaluation and alignment. However, the trade-off between evaluation accuracy and inference effici

arXiv:2608.21833v1 Announce Type: new Abstract: Recent large language models (LLMs) can operate as coding agents that build complete games from natural language requests. Game development is especiall

arXiv:2608.21794v1 Announce Type: cross Abstract: GUI grounding evaluations that expose UI elements as text metadata often treat high instruction-element embedding similarity as evidence of semantic g

arXiv:2601.20996v2 Announce Type: replace Abstract: Existing benchmarks for computational materials discovery primarily evaluate static predictive tasks or isolated computational sub-tasks. While valu

arXiv:2608.22678v1 Announce Type: cross Abstract: UAV vision-language navigation (UAV-VLN) is commonly evaluated as goal reaching, but inspection-oriented deployment requires the agent to stop within

arXiv:2608.21097v1 Announce Type: new Abstract: LLM-as-Judge systems can produce multi-dimensional evaluations, such as trustworthiness, reliability, and factuality, and these outputs are often interp

arXiv:2608.20320v1 Announce Type: new Abstract: Travel behavior research increasingly combines digital data collection with predictive modeling, yet these stages are often developed and evaluated sepa

arXiv:2608.20153v1 Announce Type: new Abstract: Large language models (LLMs) have shown growing potential for automated theoretical computer science (TCS) research, yet existing benchmarks remain far

arXiv:2608.19475v1 Announce Type: cross Abstract: This work introduces a formal semantic-block model for specifications and an execution-judged benchmark for evaluating specification quality independe

arXiv:2608.20169v1 Announce Type: cross Abstract: We present a novel approach to efficient LLM agent harness optimization through adaptive validation task selection. Harness optimization iteratively r

arXiv:2603.19997v2 Announce Type: replace Abstract: We investigate the separation of literal interpretation from contextual inference in a collaborative block-building tasks, where an agent must resol

arXiv:2608.18307v1 Announce Type: new Abstract: Current evaluation of computer-use agents is split between long-horizon workflow benchmarks and atomic GUI-grounding tests. This leaves an under-instrum

arXiv:2608.18336v1 Announce Type: new Abstract: When evaluating language models on human exams, benchmarks typically score each response as right or wrong and report the overall accuracy. This approac

arXiv:2608.18081v1 Announce Type: new Abstract: Artificial agentic systems increasingly operate as behavioral systems by interacting with dynamic environments, pursuing goals, and adapting over time.

arXiv:2608.17994v1 Announce Type: new Abstract: Using LLMs as judges has become standard practice for evaluating model outputs at scale. This is particularly common for subjective, open-ended tasks su

arXiv:2608.17299v1 Announce Type: new Abstract: Time Series Foundation Models (TSFMs) have recently emerged as a highly promising paradigm for cross-domain zero-shot forecasting. However, existing eva

arXiv:2510.06039v2 Announce Type: replace-cross Abstract: Reliable evaluation of knowledge-grounded Large Language Models (LLMs) in Chinese requires resources that explicitly align Chinese-language te

arXiv:2608.14684v1 Announce Type: cross Abstract: LLM judges increasingly evaluate responses against fine-grained rubric checklists. When a sample requires multiple rubrics, current methods typically

arXiv:2608.16480v1 Announce Type: cross Abstract: We present RISE (Roadside Infrastructure Sequence Understanding and Evaluation), a framework spanning metric 3D tracking and structured vision-languag

arXiv:2602.09992v2 Announce Type: replace-cross Abstract: Several recent contributions have evaluated the Poverty of the Stimulus Hypothesis (PoSH) using Artificial Neural Networks (ANNs). The results