Search: “evaluation”

arXiv:2609.28761v1 Announce Type: cross Abstract: We design, implement, and evaluate KathDB-FAO, a new query evaluation subsystem for our KathDB multimodal DBMS. KathDB-FAO takes as input a query in n

arXiv:2603.10287v2 Announce Type: replace-cross Abstract: LLM-as-a-Judge is a flexible framework for text evaluation, which allows us to obtain scores for the quality of a given text from various pers

arXiv:2609.28605v1 Announce Type: cross Abstract: The Factivity Inference Evaluation 2026 (FIE2026) classifies Chinese context-hypothesis pairs into nine ordered factivity intervals. Its evaluation me

arXiv:2609.25689v1 Announce Type: new Abstract: Recent advances in learning-based robot policies have demonstrated promising progress, yet they are predom- inantly evaluated in static or quasi-static

arXiv:2609.22793v1 Announce Type: new Abstract: LLM-based machine translation evaluation can closely match human judgments, but in practice it remains largely diagnostic, with the signals rarely trans

arXiv:2609.24308v1 Announce Type: new Abstract: Evaluating world models requires assessing both the quality of the worlds they generate and their consistency and responsiveness under exploration, inte

arXiv:2609.23010v1 Announce Type: new Abstract: Iterative text-to-motion generation delivers high-quality and semantically aligned motions but requires multiple network evaluations, resulting in subst

arXiv:2609.21267v1 Announce Type: new Abstract: Production LLM agents are evaluated repeatedly as they evolve, but full agent benchmarks are costly to rerun. We study efficient recurring evaluation fo

arXiv:2609.20124v1 Announce Type: cross Abstract: Evaluating live streaming speech synthesis (TTS) requires assessing fine-grained, highly expressive prosody such as emotion, intonation, and energy wh

arXiv:2609.17985v1 Announce Type: new Abstract: AI agents are usually evaluated by whether they complete a task. In interactive service settings, a successful agent can still frustrate users by asking

arXiv:2609.19072v1 Announce Type: new Abstract: Large language models are increasingly used for content moderation, but most evaluations still report aggregate accuracy on individual benchmarks. We in

arXiv:2609.18525v1 Announce Type: new Abstract: Modern evaluation of learning pipelines often reduces to downstream accuracy, leaving open the question of why tasks succeed or fail. TriProbe addresses

arXiv:2609.16023v1 Announce Type: new Abstract: Medical evaluation is shifting from static option-based questioning to realistic clinical scenarios with open-ended output modes. Grading these at scale

arXiv:2609.17112v1 Announce Type: new Abstract: Large video models have exhibited impressive performance on a wide range of visual question answering tasks, owing to the rise of powerful, pretrained t

arXiv:2609.16878v1 Announce Type: cross Abstract: Despite its crucial role in video object removal (VOR), existing evaluation paradigms face two critical limitations: questionable references and a mis

arXiv:2609.13168v1 Announce Type: cross Abstract: Speech AI, any AI system that recognizes, transforms, or generates speech, is built and evaluated across two communities with only a small overlap: te

arXiv:2609.14739v1 Announce Type: cross Abstract: Large language models are increasingly used in high-stakes domains such as law, where systems must ground their reasoning in retrieved evidence and ab

arXiv:2609.13454v1 Announce Type: cross Abstract: Clinical decisions are prospective, but clinical language models are often evaluated on retrospective records that reveal the final diagnosis, treatme

arXiv:2609.15640v1 Announce Type: cross Abstract: Long-text image--text congruence scoring is increasingly important for vision-language systems that must evaluate whether detailed textual description

arXiv:2609.13637v1 Announce Type: new Abstract: Persistent agents need evaluations that distinguish identity facts they can recall from those they express and enact. We introduce PAI-Bench, a provider

arXiv:2609.15794v1 Announce Type: new Abstract: Clinical AI systems are evaluated with instruments built for research settings (reference-based similarity metrics and expert rubric panels) that measur

arXiv:2609.14057v1 Announce Type: new Abstract: Frontier LLMs are increasingly capable of conducting automated research, yet their creativity in this setting has not been systematically evaluated. In

arXiv:2609.15939v1 Announce Type: cross Abstract: Language-model agents increasingly operate over complete software repositories, yet cybersecurity evaluations primarily measure whether they can detec

arXiv:2609.12524v1 Announce Type: cross Abstract: We study the strategic design of a two-echelon spare-parts inventory network where evaluating each candidate topology requires an expensive inventory

arXiv:2609.11897v1 Announce Type: new Abstract: Causal discovery aims to uncover causal structures from data and is fundamental to scientific reasoning and intervention-based decision making. Its eval

arXiv:2601.08536v3 Announce Type: replace Abstract: Deep Research Agents (DRA) aim to help users search the web, synthesize information, and deliver comprehensive investigative reports. Prior benchmar

arXiv:2609.09678v1 Announce Type: new Abstract: Clinical diagnosis agents must decide not only what test to request next, but also when to diagnose or defer. Existing agent benchmarks largely evaluate

arXiv:2609.07075v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) is commonly evaluated by whether the final answer is correct. That test is insufficient: an answer can match its re

arXiv:2405.02228v4 Announce Type: replace-cross Abstract: Large language models (LLMs) increasingly generate citation-backed responses, yet citation hallucination remains a major challenge for trustwo

arXiv:2609.07153v1 Announce Type: cross Abstract: Minimal-pair benchmarks such as BLiMP evaluate linguistic knowledge by testing whether language models (LMs) prefer acceptable sentences over minimall

arXiv:2609.07309v1 Announce Type: cross Abstract: Robustness evaluation of large language models (LLMs) remains a critical challenge, particularly in assessing their sensitivity to perturbations in in

arXiv:2609.09250v1 Announce Type: cross Abstract: A verifier for robot policies reads a candidate behavior and returns a score for how well it did, used both to evaluate vision-language-action policie

arXiv:2609.07601v1 Announce Type: cross Abstract: The application of large language models (LLMs) to personalized medical assistants has garnered growing interest. However, existing medical benchmarks

arXiv:2609.08322v1 Announce Type: cross Abstract: Multilingual LLMs show stereotype-related behavior that varies across languages, but behavioral scores do not show where the relevant information is r

arXiv:2603.27611v2 Announce Type: replace Abstract: When a cognitive system modifies its own functioning, what exactly does it modify: a low-level rule, a control rule, or the criterion that evaluates

arXiv:2609.06535v1 Announce Type: new Abstract: We introduce 3DHarnessBench, a benchmark that evaluates the agentic ability of frontier vision-language models (VLMs) to recover 3D geometry as Blender

arXiv:2609.07596v1 Announce Type: cross Abstract: Vision-language models are increasingly used in settings where some input modalities may be unavailable, yet we know little about whether they can fai

arXiv:2609.06207v1 Announce Type: new Abstract: Image-to-video generators are often credited with absorbing physical dynamics as implicit world models, a claim the community currently checks with plau

arXiv:2508.11431v3 Announce Type: replace Abstract: An object can disappear from a 3D scene, yet still be detectable. Even after visual removal, modern vision models may infer what was originally pres

arXiv:2609.06232v1 Announce Type: new Abstract: Ground-truth defect masks in industrial inspection datasets are typically reserved for evaluation. This paper repurposes them as spatial supervision sig

arXiv:2609.05009v1 Announce Type: new Abstract: Safety evaluations can mischaracterize deployed behaviour if artificial-intelligence systems respond to being evaluated. We test this possibility in a f

arXiv:2609.04173v1 Announce Type: new Abstract: For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases.

arXiv:2608.29632v1 Announce Type: cross Abstract: Competitive programming is increasingly being used to evaluate the algorithmic reasoning capabilities of large language models (LLMs). However, existi

arXiv:2609.00918v1 Announce Type: new Abstract: Large language models are increasingly used as interactive recommender assistants. Their evaluation should therefore go beyond plausible item recommenda

arXiv:2608.29463v1 Announce Type: cross Abstract: A benchmark score is a joint property of the model, the evaluation harness, the elicitation budget, the sampled population, and contamination status.

arXiv:2608.30568v1 Announce Type: cross Abstract: Background: Population level assessments of predictive artificial intelligence (AI) can conceal performance disparities across subgroups. Fairness eva