Search: “evaluation”

arXiv:2604.09294v1 Announce Type: new Abstract: Dexterity is a central yet ambiguously defined concept in the design and evaluation of anthropomorphic robotic hands. In practice, the term is often use

arXiv:2602.11354v2 Announce Type: replace Abstract: The literature has witnessed an emerging interest in AI agents for automated assessment of scientific papers. Existing benchmarks focus primarily on

arXiv:2604.07814v1 Announce Type: new Abstract: Accurate and interpretable plant disease diagnosis remains a major challenge for vision-language models (VLMs) in real-world agriculture. We introduce A

arXiv:2506.01062v4 Announce Type: replace Abstract: We introduce SealQA, a new challenge benchmark for evaluating SEarch-Augmented Language models on fact-seeking questions where web search yields con

arXiv:2512.20159v2 Announce Type: replace-cross Abstract: Code from LLM-assisted development may contain defects like API hallucinations, motivating the development of code evaluation metrics to asses

arXiv:2610.19906v1 Announce Type: new Abstract: LLMs have achieved strong performance on chemical prediction benchmarks. However, existing evaluation methods typically provide target entity identity i

arXiv:2604.06138v2 Announce Type: replace-cross Abstract: Long-context audio reasoning is underserved in both training data and evaluation. Existing benchmarks target short-context tasks, and the open

arXiv:2610.19719v1 Announce Type: cross Abstract: Scientific taste shapes which research directions researchers pursue, and it varies across individuals. Existing work evaluates research directions al

arXiv:2610.20027v1 Announce Type: cross Abstract: Linters are widely used in code maintenance and real-time IDE editing to surface lint alerts related to performance, style, error-prone, and other pat

arXiv:2607.08964v3 Announce Type: replace Abstract: AI agents have become capable of autonomously completing short, well-specified tasks. However, existing terminal benchmarks largely focus on simple

arXiv:2601.22050v2 Announce Type: replace Abstract: In recent years, multilingual Large Language Models (LLMs) have become an inseparable part of daily life, making it crucial for them to master the r

arXiv:2610.19262v1 Announce Type: new Abstract: Robots following persistent language instructions must decide which cached room judgments remain valid as their environment changes. We present MIND (Ma

arXiv:2610.15282v1 Announce Type: cross Abstract: Large Language Models (LLMs) have made substantial progress in multilingual question answering, yet their ability to understand low-resource socioling

arXiv:2610.19063v1 Announce Type: new Abstract: As video models enable longer and more realistic interactive experiences, maintaining a world consistent with its history becomes essential. This requir

arXiv:2610.16491v1 Announce Type: new Abstract: Improving an agent through successive revisions requires balancing exploration and evaluation under a limited budget. A narrow search can miss useful se

arXiv:2610.16627v1 Announce Type: cross Abstract: Existing human-centred XAI research predominantly evaluates explanations after they have been generated. However, this fails to provide actionable ins

arXiv:2610.17685v1 Announce Type: cross Abstract: Enterprise agents face safety risks that cannot be determined from individual actions alone. Existing safety benchmarks largely evaluate action-local

arXiv:2610.18829v1 Announce Type: new Abstract: With the rapid development of multimodal large language models (MLLMs), evaluating their video understanding capabilities has become increasingly import

arXiv:2610.13335v1 Announce Type: new Abstract: Tool-using agents increasingly require a common substrate for building, training, evaluating, distilling, and maintaining agentic systems, yet existing

arXiv:2610.12954v1 Announce Type: new Abstract: This paper introduces MathVis, a benchmark designed to evaluate LLMs' ability to visualize frontier mathematical problems and perform diagram-assisted r

arXiv:2610.13563v1 Announce Type: cross Abstract: Repository-level issue-resolution benchmarks such as SWE-bench evaluate submitted patches primarily based on whether they pass the correctness tests,

arXiv:2610.13617v1 Announce Type: new Abstract: User simulators are gaining attention as environments to train LLM assistants that can better cope with the nuances of multi-turn conversations with rea

arXiv:2610.10918v1 Announce Type: new Abstract: Large Language Models (LLMs) are considered to be a more efficient and cost-effective alternative to human judgment for Machine Translation (MT) evaluat

arXiv:2610.12423v1 Announce Type: new Abstract: We present Pumpire, a unified benchmark for evaluating metric point-pair distance estimation capability of both image- and video-level 3D foundation mod

arXiv:2610.09633v1 Announce Type: cross Abstract: The evaluation of coding agents generally strives to be as realistic as possible. In our study, we collect 4,782 agent sessions of real software engin

Introducing Open SLM Evaluations! Open SLM Evaluations is a dataset containing benchmarks for 106 SLMs(150M parameters and below). It has benchmark scores for each model, links, parameter counts, and

arXiv:2610.08319v1 Announce Type: new Abstract: In proof assistants such as Lean, a generated proof must pass machine compilation checks, so evaluation needs no human scoring. Direct generation fails

arXiv:2610.08791v1 Announce Type: new Abstract: Video world models can produce visually convincing yet physically inconsistent sequences, raising concerns about their reliability for prediction and pl

arXiv:2610.04381v1 Announce Type: new Abstract: Fine-grained image editing requires more than producing a visually plausible result: an editor must execute the requested attribute change precisely whi

arXiv:2610.04087v1 Announce Type: new Abstract: Agentic workflows are increasingly used across domains such as technology, finance, and enterprise operations. As these agents become more widely deploy

arXiv:2610.05790v1 Announce Type: new Abstract: Remote sensing foundation models (RSFMs) are commonly evaluated using aggregate metrics, which can hide systematic performance disparities across ecolog

arXiv:2610.05775v1 Announce Type: new Abstract: A video assistant must speak when its instruction warrants a response and stay silent otherwise. We introduce a benchmark that evaluates this decision f

arXiv:2610.02571v1 Announce Type: cross Abstract: As AI-assisted programming becomes increasingly mainstream, the environmental impact of AI-generated software has emerged as an important consideratio

arXiv:2610.02608v1 Announce Type: new Abstract: Time series forecasting (TSF) increasingly drives decisions in transportation, energy, finance, healthcare, and infrastructure, yet current evaluation r

arXiv:2610.02617v1 Announce Type: cross Abstract: Evaluating WebUI code generation at scale is difficult: outputs may compile and look plausible yet fail under user interaction, and prior benchmarks l

arXiv:2610.00834v1 Announce Type: new Abstract: ARC-AGI-3 evaluates agents in interactive environments whose rules and objectives must be inferred from observation. We present Kepler, an open-source h

arXiv:2610.01037v1 Announce Type: new Abstract: Optimizing merging coefficients for large language models can require many costly benchmark evaluations. We propose extbf{Simplex-Lattice Interpolation

arXiv:2609.38781v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) offer a promising approach for recovering numerical data from scientific charts, but their ability to reconst

arXiv:2609.40297v1 Announce Type: new Abstract: Autonomous exploration demands that robots continuously evaluate candidate viewpoints based on their expected information gain and execution cost. Sampl

arXiv:2609.05797v3 Announce Type: replace Abstract: Safety monitors are evaluated by recall on harmful prompts, regardless of whether the target model would answer them. Yet a monitor matters most on

Agents built and deployed on MongoDB’s Atlas Agent Engine can export OpenTelemetry traces to Arize AX, where teams can inspect each run, evaluate both the outcome and execution path, and test fixes be

arXiv:2609.34113v2 Announce Type: replace Abstract: Understanding where and why Graphical User Interface (GUI) agents fail is essential for building more reliable systems, yet current evaluation relie

arXiv:2609.37515v1 Announce Type: cross Abstract: Thorough evaluation of vision-language models (VLMs) has become prohibitively expensive, as benchmarks span an ever-broader spectrum of capabilities a

arXiv:2609.35269v1 Announce Type: cross Abstract: The rising number of concept unlearning techniques for text-to-image (T2I) diffusion models has produced a fragmented evaluation landscape. Methods ar

arXiv:2609.33158v1 Announce Type: new Abstract: Progress in AI-based retinal image analysis has advanced with foundation models, yet evaluating their reliability remains challenging. Performance repor

arXiv:2609.30897v1 Announce Type: new Abstract: How should we evaluate language models when more than one interpretation can be right? Cultural benchmarks often test factual knowledge, agreement with