Search: “evaluation”

arXiv:2609.08592v1 Announce Type: new Abstract: Evaluating tool-augmented LLM agents requires diverse, realistic user inputs yet most evaluation frameworks use flat role descriptions ('you are an angr

arXiv:2609.03276v1 Announce Type: new Abstract: Evaluating robot manipulation policies is becoming increasingly important as generalist models, particularly vision-language-action (VLA) models, are de

arXiv:2502.00857v2 Announce Type: replace Abstract: Large Language Models (LLMs) increasingly provide direct answers to user questions, raising concerns about reduced engagement in critical thinking a

arXiv:2608.13566v1 Announce Type: cross Abstract: Post-training papers, model cards, and blog posts often treat scores on a small set of coding benchmarks (e.g., SWE-bench and LiveCodeBench) as eviden

arXiv:2603.02565v2 Announce Type: replace-cross Abstract: The Generator-Evaluator (G-E) framework generates K candidate sequences and uses an evaluator to select the highest-scoring one, which is wide

arXiv:2503.06437v3 Announce Type: replace-cross Abstract: We present SEED (Semantic Evaluation for Visual Brain Decoding), a novel metric for evaluating the semantic decoding performance of visual bra

arXiv:2607.12835v1 Announce Type: new Abstract: Rubric-based evaluation is a promising approach for assessing open-ended outputs from LLM-based research agents, particularly in paper reproduction, whe

arXiv:2602.06486v4 Announce Type: replace Abstract: Evaluating agentic AI on open-ended professional tasks faces a fundamental dilemma between rigor and flexibility. Static rubrics provide rigorous, r

arXiv:2606.20820v1 Announce Type: new Abstract: Can we trust evaluation scores to capture an LLM's true real-world performance? Certifiable evaluation answers this question by providing guarantee for

arXiv:2605.27132v1 Announce Type: new Abstract: Multilevel image thresholding is widely used for segmentation in applications ranging from medical imaging to remote sensing. Classical objective functi

arXiv:2605.17382v1 Announce Type: new Abstract: The rapid progress of generative artificial intelligence has exposed fundamental limitations in existing evaluation methodologies, particularly for open

arXiv:2604.21214v1 Announce Type: cross Abstract: Text-to-SQL models have significantly improved with the adoption of Large Language Models (LLMs), leading to their increasing use in real-world applic

arXiv:2511.18825v3 Announce Type: replace Abstract: Evaluating AI-generated video (AIGV) quality hinges on three crucial dimensions: visual quality, dynamic quality, and text-video alignment. While nu

arXiv:2610.00651v1 Announce Type: new Abstract: Agent evaluations are increasingly used to compare LLMs and inform deployment decisions, yet ranks can reflect not only the model but also the effects o

arXiv:2609.36264v1 Announce Type: new Abstract: Reliable evaluation of large language models (LLMs) is essential for their development and deployment, yet is often costly, risky, and difficult to perf

arXiv:2609.21231v1 Announce Type: new Abstract: Reference-based metrics for Grammatical Error Correction (GEC) such as M^2 and ERRANT assume that the reference set enumerates all valid edits, and ther

arXiv:2609.09875v1 Announce Type: new Abstract: Existing evaluation frameworks mostly assess only one part of AI agents, such as task completion (AgentBench) or security robustness (AgentDojo, ASB), r

arXiv:2608.29478v1 Announce Type: cross Abstract: Scholarly work which aims to describe potential societal impacts (e.g., risks) of proliferating technology (especially related to artificial intellige

arXiv:2608.28725v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as graders, verifiers, and process auditors, but most mathematical evaluations still emphasize final-

arXiv:2608.26159v1 Announce Type: cross Abstract: Self-Generated Text Recognition (SGTR)--the ability of an LLM to identify its own outputs--poses risks to AI safeguards that rely on LLMs as evaluator

arXiv:2608.24258v1 Announce Type: new Abstract: AI systems are increasingly evaluated for legally accountable settings, where correct outputs must also be justifiable against an applicable legal stand

arXiv:2608.21057v1 Announce Type: new Abstract: Agentic large language model (LLM) systems are reshaping scientific workflows in chemistry and drug discovery, but evaluating their open-ended, tool-aug

arXiv:2608.07537v1 Announce Type: cross Abstract: In this study, we propose a framework that incorporates subjective evaluations provided by a Vision-Language Model (VLM) into the fitness evaluation a

arXiv:2608.07523v1 Announce Type: cross Abstract: Difficulty differences across parallel-class programming examinations affect the fairness of course assessment. This study repositions large language

arXiv:2608.00100v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly being evaluated for medical imaging, but many available benchmarks emphasize disease classification, repo

arXiv:2607.18999v2 Announce Type: replace-cross Abstract: Evaluating multi-turn medical consultation agents requires judging the diagnostic support provided by the histories they elicit through intera

arXiv:2606.29784v1 Announce Type: cross Abstract: Reliable generative AI models critically rely on expert human annotations to evaluate output quality, yet these 'gold' labels are expensive to collect

arXiv:2606.11079v1 Announce Type: new Abstract: Evaluation remains a critical bottleneck for interactive agent development. Existing evaluation methods often rely on static benchmarks, which fail to c

arXiv:2605.17439v1 Announce Type: cross Abstract: Evaluating LLM-generated interactive software requires execution in addition to static analysis. The key difficulty is that correctness is a graph-lev

arXiv:2605.13223v1 Announce Type: new Abstract: Text-to-image (T2I) generation has advanced rapidly, making reliable evaluation critical as performance differences between models narrow. Existing eval

arXiv:2605.11496v1 Announce Type: cross Abstract: Recent published evidence from frontier laboratories shows that contemporary AI models can recognise evaluation contexts, latently represent them, and

arXiv:2512.16055v2 Announce Type: replace Abstract: Safety-critical corner cases, difficult to collect in the real world, are crucial for evaluating end-to-end autonomous driving. Adversarial interact

arXiv:2604.16310v1 Announce Type: cross Abstract: Evaluating Retrieval-Augmented Generation (RAG) systems using static multi-turn datasets fails to capture the dynamic nature of real-world dialogues.

arXiv:2603.13966v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models are increasingly evaluated across multiple simulation benchmarks, yet adding each benchmark to an evaluation pip

arXiv:2610.18699v1 Announce Type: new Abstract: Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world mod

arXiv:2610.17521v1 Announce Type: new Abstract: Evaluating VLA policies on real robots is costly, time-consuming, and difficult to reproduce, making real-to-sim simulation essential for scalable evalu

arXiv:2610.09872v1 Announce Type: cross Abstract: Evaluating agents by outcomes alone can obscure the capabilities that produce them. This problem is especially pronounced in evolving environments, wh

arXiv:2609.40103v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as automated judges for AI-generated content, yet a single judge is unreliable and even a panel o

arXiv:2609.34799v2 Announce Type: replace Abstract: Language-conditioned trajectory generation is here, but its evaluation has not kept pace. Existing pedestrian trajectory metrics compare trajectorie

arXiv:2609.32547v1 Announce Type: new Abstract: Large language models are commonly evaluated by generating a small number of stochastic responses for each prompt in a benchmark. Because inference budg

arXiv:2609.22119v1 Announce Type: new Abstract: Evaluation awareness poses an unprecedented threat to model evaluation, but the mechanisms by which models detect it remain unknown. This study focuses

arXiv:2603.17165v2 Announce Type: replace-cross Abstract: We present SAL (SLAM Adversarial Lab), a modular framework for evaluating visual SLAM systems under adversarial conditions such as fog and rai

arXiv:2603.15976v2 Announce Type: replace Abstract: While LLMs have accelerated scientific code generation, comprehensively evaluating generated code remains challenging. Many benchmarks emphasize fun

arXiv:2609.05439v1 Announce Type: new Abstract: Current evaluation methods for large language models are coarse-grained and decoupled from generation, producing generic explanations that fail to provi

arXiv:2609.06059v1 Announce Type: new Abstract: As large language models evolve from question-answering systems into general-purpose agents, evaluation must move beyond static answer correctness to as

arXiv:2609.02942v1 Announce Type: cross Abstract: LLM-as-a-Judge pipelines are increasingly used to evaluate AI-generated text, based on the assumption that judgments arise from reasoning over candida