Search: “evaluation”

arXiv:2605.21063v1 Announce Type: new Abstract: Typical LLM responses tend to follow a default style, even though users often have distinct preferences regarding tone, verbosity, and formality that th

arXiv:2605.21443v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly being explored for video game quality assurance, especially gameplay glitch detection. Most existing eval

arXiv:2603.28675v2 Announce Type: replace Abstract: Facial recognition systems are increasingly deployed in law enforcement and security contexts, where algorithmic decisions can carry significant soc

arXiv:2605.19855v1 Announce Type: cross Abstract: Concept-based Explainable Artificial Intelligence (XAI) interprets deep learning models using human-understandable visual features (e.g., textures or

arXiv:2605.18827v1 Announce Type: cross Abstract: Multiple-choice QA benchmarks usually evaluate small language models (SLMs) as direct answerers, but deployed language-model systems increasingly rely

New course: Build AI agents that generate images and videos -- an under-explored frontier. A key to performance is having the agent evaluate its own output, and iterate to improve quality. This short

you can condense long horizon evals with agents into smaller subsets that still let you test intended behavior. i'm currently evaluating an agent that runs for 30+ minutes, and analyzes thousands of t

arXiv:2511.23253v3 Announce Type: replace Abstract: Recent advancements in Vision-Language Models (VLMs) have significantly impacted various industries. In agriculture, these multimodal capabilities h

arXiv:2605.18667v1 Announce Type: new Abstract: Earth embedding models transform Earth observation data into embeddings uniquely tied to locations on the Earth's surface. These models are typically ev

arXiv:2605.17370v1 Announce Type: new Abstract: Cognitive behavioural therapy is widely used to help patients understand and manage psychological distress. It is often delivered through spoken convers

arXiv:2605.18498v1 Announce Type: cross Abstract: Expert specialization in Mixture-of-Experts (MoE) models remains poorly understood, with traditional evaluations conflating architectural load-balanci

arXiv:2502.04055v2 Announce Type: replace Abstract: Current evaluations of synthetic tabular data mainly focus on how well joint distributions are modeled, often overlooking the assessment of their ef

arXiv:2605.18663v1 Announce Type: new Abstract: As LLM benchmarks saturate, the evaluation community has pursued two strategies to increase difficulty: escalating knowledge demands (GPQA, HLE) or remo

arXiv:2605.17932v1 Announce Type: cross Abstract: Prompt compression reduces inference cost and context length in large language models, but prior evaluations focus primarily on autoregressive archite

arXiv:2605.14488v1 Announce Type: new Abstract: Large Language Models (LLMs) augmented with Retrieval-Augmented Generation (RAG) techniques are revolutionizing applications across multiple domains, su

arXiv:2605.10186v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly integrated into legal drafting and research workflows, where incorrect citations or fabricated precedent

arXiv:2605.10820v1 Announce Type: new Abstract: Scientific discovery is fundamentally a resource-constrained process that requires navigating complex trade-offs between the quality and quantity of mea

arXiv:2605.08437v1 Announce Type: cross Abstract: Existing benchmarks for legal AI focus primarily on tasks where LLMs must produce legal arguments or documents, yet the capacity to judge such argumen

arXiv:2605.07926v1 Announce Type: new Abstract: As LLM-based agents increasingly rely on external tools, it is important to evaluate their ability to sustain tool-grounded reasoning beyond familiar wo

arXiv:2605.07046v1 Announce Type: cross Abstract: Evaluation of large language models (LLMs) is increasingly critical, yet standard benchmarking methods rely on average accuracy, overlooking both the

arXiv:2605.07251v1 Announce Type: new Abstract: Large Language Models (LLMs) have become increasingly capable as tool-using agents, with benchmarks spanning diverse general agentic tasks. Yet rigorous

arXiv:2512.15567v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly applied to scientific research, yet prevailing science benchmarks probe decontextualized knowledge and

arXiv:2605.07675v1 Announce Type: new Abstract: We introduce FactoryBench, a benchmark for evaluating time-series models and LLMs on machine understanding over industrial robotic telemetry. Q&A pairs

arXiv:2605.04018v1 Announce Type: new Abstract: Reasoning-intensive retrieval aims to surface evidence that supports downstream reasoning rather than merely matching topical similarity. This capabilit

arXiv:2409.14500v5 Announce Type: replace Abstract: Although data that can be naturally represented as graphs is widespread in real-world applications across diverse industries, popular graph ML bench

arXiv:2604.27470v1 Announce Type: new Abstract: Millions of clinicians use ChatGPT to support clinical care, but evaluations of the most common use cases in model-clinician conversations are limited.

arXiv:2604.25359v1 Announce Type: new Abstract: Large Language Models are increasingly being deployed to extract structured data from unstructured and semi-structured sources: parsing invoices, medica

arXiv:2509.20374v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated strong performance across general NLP tasks, but their utility in automating numerical experime

arXiv:2406.10185v2 Announce Type: replace Abstract: Large Vision Language Models (LVLMs) are increasingly integral to healthcare applications, including medical visual question answering and imaging r

arXiv:2604.23347v1 Announce Type: new Abstract: We present a comprehensive evaluation of Large Language Models (LLMs) on Computer Science (CS) Data Structure examination questions. Our work introduces

arXiv:2604.23813v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable performance in Visually Rich Document Understanding (VRDU) tasks, but their capabili

Great paper on improving proactive agents. (bookmark it) Proactive agents act before you do. But how do you evaluate something that's supposed to anticipate needs you haven't expressed? This work intr

arXiv:2604.21006v1 Announce Type: new Abstract: We introduce Deep FinResearch Bench, a practical and comprehensive evaluation framework for deep research (DR) agents in financial investment research.

arXiv:2604.21916v1 Announce Type: new Abstract: As frontier language models attain near-ceiling performance on static mathematical benchmarks, existing evaluations are increasingly unable to different

arXiv:2508.17761v3 Announce Type: replace Abstract: In safety-critical applications data-driven models must not only be accurate but also provide reliable uncertainty estimates. This property, commonl

arXiv:2604.18660v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used in education, yet their default helpfulness often conflicts with pedagogical principles. Prior work

arXiv:2604.19510v1 Announce Type: new Abstract: Variability in illumination is a primary factor limiting deep learning robustness for field-based plant disease detection. This study evaluates Histogra

arXiv:2604.19526v1 Announce Type: cross Abstract: Cross-site scripting (XSS) remains a persistent web security vulnerability, especially because obfuscation can change the surface form of a malicious

arXiv:2604.19092v1 Announce Type: cross Abstract: Recent advances in large-scale video world models have enabled increasingly realistic future prediction, raising the prospect of leveraging imagined v

arXiv:2510.19028v3 Announce Type: replace Abstract: As LLMs are increasingly deployed in real-world interactions, their social reasoning in interpersonal communication becomes critical. To explore the

arXiv:2508.17458v2 Announce Type: replace Abstract: Verbal multiword expressions (VMWEs) remain difficult for machine translation because their meanings are often not recoverable from their component

arXiv:2602.06663v2 Announce Type: replace Abstract: Unified multimodal models (UMMs) have shown impressive capabilities in generating natural images and supporting multimodal reasoning. However, their

arXiv:2510.09351v2 Announce Type: replace Abstract: While Small Language Models (SLMs) have demonstrated promising performance on an increasingly wide array of commonsense reasoning benchmarks, curren

EDINET-Bench is a benchmark dataset that evaluates large language models on complex financial analysis tasks using Japanese financial statements from the EDINET database. The benchmark tests LLM capab

arXiv:2604.16279v1 Announce Type: new Abstract: Large Language Models (LLMs) have the potential to accelerate small molecule drug design due to their ability to reason about information from diverse s

arXiv:2604.06484v2 Announce Type: replace Abstract: Cultural values are expressed not only through language but also through visual scenes and everyday social practices. Yet existing evaluations of cu