Search: “evaluation”

arXiv:2610.15994v1 Announce Type: new Abstract: Representing individual-level behavioral heterogeneity is challenging in epidemiological modeling, particularly when empirical behavioral data are spars

arXiv:2610.19848v1 Announce Type: new Abstract: We present OpenTWBench, an open evaluation suite for Taiwan-domain knowledge in large language models, built around two benchmarks. tw-exam-bench contai

arXiv:2610.17490v1 Announce Type: new Abstract: Retrospective evaluation of early-warning models may not reflect information available when predictions are issued and outcomes become observable. We de

arXiv:2610.16008v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems are increasingly used for biomedical Question Answering (QA). However, evaluating their effectiveness rem

arXiv:2610.14550v1 Announce Type: cross Abstract: Dynamic Product Advertising (DPA) retrieval systems are a core component of modern e-commerce advertising; however, their evaluation remains challengi

arXiv:2610.12458v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are rapidly evolving toward continuous audio--visual reasoning, creating an urgent need for evaluations that ex

arXiv:2610.11669v1 Announce Type: new Abstract: Holistic and semantics-aware co-speech gesture generation has advanced rapidly, yet evaluation remains behind: objective metrics do not consistently ref

arXiv:2610.11766v1 Announce Type: cross Abstract: Safety evaluations of large language models commonly summarize harmful-output behavior with attack success rate (ASR). Yet the same non-harmful outcom

arXiv:2610.10368v1 Announce Type: cross Abstract: Adaptive computation aims to improve language-model inference by tailoring execution to each input. For layer programs, oracle evaluations use known a

arXiv:2610.09569v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for emotional support, raising concern that sustained use may draw users away from their real-world r

arXiv:2610.06854v1 Announce Type: cross Abstract: Synthetic data generation is critical for privacy compliance, machine learning augmentation, and software testing. While single-table evaluation is we

arXiv:2610.07722v1 Announce Type: new Abstract: Activation steering provides a lightweight and flexible way to control large language model (LLM) behavior. However, effective steering requires more th

arXiv:2610.07362v1 Announce Type: new Abstract: We evaluate large language models (LLMs) in multi-turn interactions through their time-to-event: the number of interaction steps required to produce an

arXiv:2610.08719v1 Announce Type: new Abstract: Reusing the scores that select a Best-of-N winner can overstate its expected reward. We study evaluation from a fixed matrix of K independent scores per

arXiv:2610.07346v1 Announce Type: new Abstract: Repeated evaluation and differentiation of Signal Temporal Logic (STL) robustness can become a computational bottleneck in robot planning and control. S

arXiv:2610.08677v1 Announce Type: new Abstract: Off-policy evaluation (OPE) for contextual bandit policies becomes challenging when action-level importance weighting incurs excessive variance. Doubly

arXiv:2610.03193v1 Announce Type: new Abstract: The application of machine learning to dermatology has grown substantially in recent years, moving beyond proof-of-concept studies toward potential appl

arXiv:2610.02959v1 Announce Type: new Abstract: Recent text-to-image models have made substantial progress in photorealism, aesthetics, and text-image alignment. Yet visually appealing images can stil

arXiv:2609.38377v1 Announce Type: new Abstract: Evaluating the physical consistency of generated videos remains a fundamental challenge. Existing approaches rely on off-the-shelf vision-language model

arXiv:2609.38463v1 Announce Type: cross Abstract: Autonomous driving planners are typically evaluated using aggregate metrics such as driving score, destination rate, and collision rate, which do not

arXiv:2609.37771v1 Announce Type: new Abstract: Simulated manipulation benchmarks are the standard tool for evaluating vision-language-action (VLA) policies and the acceleration methods that reduce th

arXiv:2609.37322v1 Announce Type: new Abstract: Rubric-based evaluation is widely used to assess LLM-based systems by decomposing response quality into task-specific scoring criteria. However, automat

arXiv:2609.36417v1 Announce Type: new Abstract: Relational in-context learning (ICL) uses labeled support examples and their linked relational context to predict labels for new queries. This creates a

arXiv:2609.35034v1 Announce Type: cross Abstract: Off-policy evaluation, which estimates evaluation policy performance from logged data, is key for recommender ranking policies. However, logged clicks

arXiv:2609.30867v1 Announce Type: new Abstract: Difference-in-differences (DID) studies are widely used to evaluate climate policy, but assessing the evidence supporting their identification assumptio

arXiv:2609.23201v1 Announce Type: new Abstract: Benchmark scores increasingly influence the development, marketing, and selection of large language models (LLMs). Yet an overall score is interpretable

arXiv:2609.25645v1 Announce Type: cross Abstract: Exhaustively evaluating every candidate LLM configuration on every benchmark item to identify a high-performing one is costly. We formulate configurat

arXiv:2609.25352v1 Announce Type: cross Abstract: Evaluation of large language models (LLMs) for safety, security, and privacy (SSP) relies heavily on static benchmarks, which suffer from score satura

arXiv:2609.24890v1 Announce Type: new Abstract: Evaluation of Computer-Use Agents (CUAs) is often limited to the final deliverables they create (at the end of hundreds of steps) and assessed with func

arXiv:2609.24586v1 Announce Type: new Abstract: Traditional evaluation metrics provides numerical values but often lack comprehensibility, hindering effective differentiation of model performances. Ou

arXiv:2609.21803v1 Announce Type: new Abstract: Contact-rich motion planning (CRMP) is essential for robotic manipulation and locomotion, yet remains computationally challenging due to combinatorial c

arXiv:2505.16180v3 Announce Type: replace-cross Abstract: Evaluating image captions requires cohesive assessment of both visual semantics and language pragmatics, which is often not entirely captured

arXiv:2609.18323v1 Announce Type: new Abstract: Recent Omni-Modal Generative Models (Omni-Models) have advanced content generation toward unified modeling of text, images, video, and audio. MiniMax-H3

arXiv:2609.16267v1 Announce Type: cross Abstract: Many operational cases are documented more than once, at different workflow stages and for different purposes, yet model evaluations normally select o

arXiv:2609.15940v1 Announce Type: new Abstract: Vision-language-action policies are typically evaluated one perturbation at a time, providing a useful diagnosis of their sensitivity to individual dist

arXiv:2609.13149v1 Announce Type: cross Abstract: For local large language model agents, active context is a scarce resource: memory capacity, prefill latency, cache growth, and service objectives all

arXiv:2601.13885v2 Announce Type: replace-cross Abstract: Computerized Adaptive Testing (CAT) has proven effective for efficient LLM evaluation on multiple-choice benchmarks, but modern LLM evaluation

arXiv:2609.15467v1 Announce Type: cross Abstract: Adding answer options can lower multiple-choice scores without improving assessment validity. Turkish MMLU Pro examines this distinction using 12,000

arXiv:2609.12191v1 Announce Type: new Abstract: Comparing and selecting task-oriented LLM agents increasingly relies on a low-cost offline evaluation gate: persona-driven LLM user-simulators converse

arXiv:2508.05830v3 Announce Type: replace Abstract: Large Language Model (LLM) studies that use language responses elicited from depression assessments to predict scores on those same assessments ofte

arXiv:2609.11770v1 Announce Type: new Abstract: Large language models are superseded every few quarters; clinical evidence takes years. We asked whether medical research is keeping pace with the syste

arXiv:2609.05818v1 Announce Type: new Abstract: We introduce ABLE, a benchmark for evaluating LLM agents' ability to use biological AI models (BAIMs), such as ProteinMPNN and AlphaFold3, in dual-use p

arXiv:2609.05527v1 Announce Type: new Abstract: Wherever a coding agent works under engineer supervision, or a clinical model assists a radiologist, the deployment question is whether to keep the huma

arXiv:2609.05441v1 Announce Type: new Abstract: Long-term memory for LLM agents is evaluated today by conversational recall benchmarks (LoCoMo, LongMemEval), which measure question answering over dial

arXiv:2609.04766v1 Announce Type: cross Abstract: Generative AI research has increasingly evaluated factuality, citation, coverage, and report structure. Yet passing such local checks does not by itse

arXiv:2605.06508v2 Announce Type: replace-cross Abstract: Research artifacts are widely shared to support reproducibility, and artifact evaluation (AE) has become common at many leading conferences. H