Search: “evaluation”

arXiv:2605.17176v1 Announce Type: new Abstract: Emotion understanding is a core capability for LLMs to interact effectively with humans, yet existing evaluation paradigms rely on discrete emotion labe

arXiv:2508.08501v3 Announce Type: replace Abstract: We introduce GVGAI-LLM, a video game benchmark for evaluating the reasoning and problem-solving capabilities of large language models (LLMs). Built

arXiv:2601.08118v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used as human simulators, both for evaluating conversational systems and for generating fine-tuning da

arXiv:2605.17500v1 Announce Type: cross Abstract: Generative text-to-image models are typically trained on large-scale web-scraped datasets that include diverse visual content such as copyrighted and

arXiv:2605.16223v1 Announce Type: cross Abstract: Generative video models are increasingly used in design animation tasks, yet no standardized evaluation framework exists for this domain. Unlike natur

arXiv:2605.14641v1 Announce Type: cross Abstract: Class attribution maps (CAMs) provide local explanations for the decisions of convolutional neural networks. While widely used in practice, the evalua

arXiv:2601.05752v3 Announce Type: replace Abstract: We introduce AutoMonitor-Bench, the first benchmark designed to systematically evaluate the reliability of LLM-based misbehavior monitors across div

arXiv:2605.08432v1 Announce Type: cross Abstract: Calibration measures whether a model's predicted confidence aligns with its empirical accuracy, and is central to the reliable deployment of large lan

arXiv:2602.04189v2 Announce Type: replace Abstract: Uncertainty evaluation is critical in scientific and engineering inverse problems. However, existing benchmarks on Diffusion Inverse Solvers (DIS) p

arXiv:2605.10246v1 Announce Type: new Abstract: AI scientist systems are increasingly deployed for autonomous research, yet their academic integrity has never been systematically evaluated. We introdu

arXiv:2605.06856v1 Announce Type: cross Abstract: Generative AI systems achieve impressive performance on standard benchmarks yet fail to deliver real-world utility, a disconnect we identify across 28

arXiv:2601.15050v4 Announce Type: replace Abstract: Current evaluation methods for Retrieval Augmented Generation (RAG) suffer from extit{factual myopia}: they relentlessly emphasize factual accuracy

arXiv:2605.07919v1 Announce Type: new Abstract: Medical vision--language models (VLMs) are usually evaluated on intact image--question pairs, but trustworthy clinical use requires a stronger property:

arXiv:2605.07461v1 Announce Type: new Abstract: Rubrics have been extensively utilized for evaluating unverifiable, open-ended tasks, with recent research incorporating them into reward systems for re

arXiv:2605.04208v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated impressive multilingual capabilities for well-resourced languages, yet their performance on low-resource

arXiv:2505.18466v2 Announce Type: replace Abstract: Evaluations of Large Language Models (LLMs) often overlook intersectional and culturally specific biases, particularly in underrepresented multiling

arXiv:2603.04601v2 Announce Type: replace-cross Abstract: Code generation has emerged as one of AI's highest-impact use cases, yet existing benchmarks measure isolated tasks rather than the complete '

arXiv:2605.01687v1 Announce Type: new Abstract: We present MultiBreak, a scalable and diverse multi-turn jailbreak benchmark to evaluate large language model (LLM) safety. Multi-turn jailbreaks mimic

arXiv:2512.01020v2 Announce Type: replace-cross Abstract: Evaluating the quality of LLM-generated reasoning traces in expert domains (e.g., law) is essential for ensuring credibility and explainabilit

arXiv:2604.28177v1 Announce Type: new Abstract: We introduce AEGIS, A holistic benchmark for Evaluating forensic analysis of AI-Generated academic ImageS. Compared to existing benchmarks, AEGIS featur

arXiv:2603.04751v2 Announce Type: replace Abstract: Integrating web search tools has significantly extended the capability of LLMs to address open-world, real-time, and long-tail problems. However, ev

arXiv:2501.03624v2 Announce Type: replace-cross Abstract: Large language models (LLMs) excel on many NLP benchmarks, but their behavior on real-world, semi-structured prediction remains underexplored.

arXiv:2604.20225v1 Announce Type: new Abstract: Evaluating the multilingual and multicultural capabilities of Large Language Models (LLMs) is essential for their global utility. However, current bench

arXiv:2601.15755v3 Announce Type: replace Abstract: Large language models are increasingly used to represent human opinions, values, or beliefs, and their steerability towards these ideals is an activ

arXiv:2604.18943v1 Announce Type: new Abstract: With the rise in capabilities of large language models (LLMs) and their deployment in real-world tasks, evaluating LLM alignment with human preferences

arXiv:2604.16706v1 Announce Type: cross Abstract: Automated evaluation of tool-using large language model (LLM) agents is widely assumed to be reliable, but this assumption has rarely been validated a

arXiv:2604.16976v1 Announce Type: new Abstract: Point cloud denoising is a fundamental and crucial challenge in real-world point cloud applications. Existing quantitative evaluation metrics for point

arXiv:2504.04927v2 Announce Type: replace-cross Abstract: As generative AI (GenAI) is increasingly applied in persona development to represent real users, understanding the implications and limitation

arXiv:2602.14517v2 Announce Type: replace Abstract: Large language models (LLMs) have achieved strong results in mathematical reasoning, and are increasingly deployed as tutoring and learning support

arXiv:2604.14188v1 Announce Type: cross Abstract: Large language models have demonstrated impressive performance across many domains of mathematics and physics. One natural question is whether such mo

arXiv:2604.11996v1 Announce Type: cross Abstract: Should we trust Large Language Models (LLMs) with high accuracy? LLMs achieve high accuracy on reasoning benchmarks, but correctness alone does not re

arXiv:2604.12994v1 Announce Type: cross Abstract: Logical vulnerabilities in software stem from flaws in program logic rather than memory safety, which can lead to critical security failures. Although

arXiv:2604.12040v1 Announce Type: cross Abstract: We present SIR-Bench, a benchmark of 794 test cases for evaluating autonomous security incident response agents that distinguishes genuine forensic in

arXiv:2603.25620v2 Announce Type: replace Abstract: Large language model (LLM)-based persona agents are rapidly being adopted as scalable proxies for human participants across diverse domains. Yet the

arXiv:2509.20868v2 Announce Type: replace-cross Abstract: Structured reasoning can improve the inference performance of large language models (LLMs), but it also introduces computational cost and cont

arXiv:2603.18019v2 Announce Type: replace Abstract: Do language model benchmarks actually measure what practitioners intend them to ? High-level metadata is too coarse to convey the granular reality o

arXiv:2604.06742v1 Announce Type: cross Abstract: Large Language Models (LLMs) are driving a shift towards intent-driven development, where agents build complete software from scratch. However, existi

arXiv:2610.20836v1 Announce Type: new Abstract: Evaluator-guided reasoning presents a local control problem at every step: trust the current answer, return a fallback, or spend more inference. We form

arXiv:2610.21604v1 Announce Type: new Abstract: Scientific inquiry involves iterative cycles of exploration, hypothesis generation, and reasoning over observations to uncover relationships in complex

arXiv:2610.16838v1 Announce Type: new Abstract: As language-model agents write code, move money and change records on behalf of users, deciding whether an action is authorized has become a central saf

arXiv:2610.14016v1 Announce Type: new Abstract: Most existing sports datasets are individual-centric or ball-centric, focusing on individual player actions while ignoring interactions among teammates.

arXiv:2610.18030v1 Announce Type: new Abstract: Closed-loop evaluation of end-to-end (E2E) driving systems typically relies on generating sensor observations and re-processing them through the percept

arXiv:2610.13534v1 Announce Type: new Abstract: Long-horizon embodied agents must maintain task-relevant information, coordinate heterogeneous robots, and adapt their decisions as execution unfolds. E

arXiv:2610.12596v1 Announce Type: new Abstract: Agentic large language models now tackle complex tasks, yet strategic reasoning over long horizons is under-studied: most agentic benchmarks are short a

arXiv:2610.12341v1 Announce Type: new Abstract: Adversarial games have driven advances from heuristic search to reinforcement learning, yet learning and adapting strategies from limited samples remain

arXiv:2610.11482v1 Announce Type: cross Abstract: Context: Large language model (LLM) agents are increasingly used as software and data-engineering assistants, yet evidence about locally deployable op