Search: “evaluation”

arXiv:2602.20294v2 Announce Type: replace-cross Abstract: Simulating real personalities with large language models requires grounding generation in authentic personal data. Existing evaluation approac

arXiv:2610.01317v1 Announce Type: new Abstract: Evaluating candidate architectures in neural architecture search (NAS) faces an inherent trade-off: on the one hand, reliable performance labels are lim

arXiv:2610.01052v1 Announce Type: new Abstract: We argue that as video generation extends to longer durations, subject consistency should be evaluated over extit{dynamic subject sets}. We therefore in

arXiv:2609.38409v1 Announce Type: new Abstract: Recent progress in large language model reasoning has been driven by benchmarks and reinforcement learning environments with automatically verifiable re

arXiv:2609.38335v1 Announce Type: cross Abstract: Coding agents powered by large language models (LLMs) are evolving from making localized code changes to developing complete software repositories. Ho

arXiv:2609.38473v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) is now the standard way to ground Large Language Models (LLMs) in external knowledge, yet the design space of ret

arXiv:2609.38867v1 Announce Type: new Abstract: Large language model (LLM) computer-use agents are typically evaluated with clean written instructions, despite speech being an increasingly popular int

arXiv:2609.37472v1 Announce Type: cross Abstract: Behavioral tests measure how a language model reads evidence. We ask whether those measurements help choose a recommendation interface. We evaluate si

arXiv:2609.35952v1 Announce Type: cross Abstract: We introduce HEAR (Human-recorded Evaluation of Audio-LLM bias by Real speakers), a large-scale, ecologically valid benchmark comprising 87k real huma

arXiv:2609.26550v3 Announce Type: replace Abstract: LLM-as-a-judge scales evaluation, but reasoning judges are slow and costly. We study JEV-as-a-Judge: evaluation with JEV, a decision-only judge that

arXiv:2609.38070v1 Announce Type: new Abstract: As the chain-of-thought reasoning capabilities of large language models improve, evaluating and calibrating their reasoning confidence is becoming incre

arXiv:2609.36525v1 Announce Type: cross Abstract: Distributed inference has become an indispensable part of deploying medical models under practical latency, memory, and throughput constraints. Althou

arXiv:2609.37918v1 Announce Type: new Abstract: Reasoning across videos requires aligning events, matching identities, comparing motion, and integrating partial observations. Evaluating these capabili

arXiv:2609.32890v1 Announce Type: new Abstract: Language models are increasingly evaluated as vulnerability detectors, and scores reported for similar models differ widely between papers. We measured

arXiv:2609.34807v1 Announce Type: new Abstract: Spatially conditioned diffusion models can embed words and contours in natural-looking images, but vision-language models (VLMs) may fail to recognize t

arXiv:2609.35130v1 Announce Type: new Abstract: Communication-efficient federated optimization commonly spends several gradient evaluations between server updates. Existing local-update methods use th

arXiv:2609.34673v1 Announce Type: new Abstract: Few-step generative models can generate high-fidelity samples within a few function evaluations. Despite this efficiency, generated samples may not exhi

arXiv:2609.34412v1 Announce Type: new Abstract: Natural-language manipulation instructions specify qualitative relations, whereas continuous controllers require state-evaluable task quantities, differ

arXiv:2609.35052v1 Announce Type: new Abstract: Video world models must preserve the visual state of the world over time, but existing evaluation protocols often rely on generated histories, video ref

arXiv:2609.31918v1 Announce Type: new Abstract: We introduce ROTE (RollOut Testing of Exact memorization), a benchmarking protocol for evaluating symbolic memorization of neural architectures. We stud

arXiv:2609.35504v1 Announce Type: new Abstract: Machine intelligence is often evaluated through abstract reasoning problems, yet many real-world problems are visual, such as arranging objects, repairi

arXiv:2609.30489v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated historic breakthroughs in general reasoning with early successes in biomedical science. However, existing

arXiv:2609.30553v1 Announce Type: new Abstract: Expensive evolutionary search does not always need an exact fitness estimate for every candidate. It often needs a reliable answer to a simpler question

arXiv:2609.30722v1 Announce Type: cross Abstract: Existing roadside traffic datasets support perception, forecasting, and visual question answering, but they do not evaluate counterfactual video gener

arXiv:2609.30140v1 Announce Type: new Abstract: In this paper, we study the joint selection of an environmental support contact and a whole-body configuration for a prescribed loco-manipulation task.

arXiv:2609.29808v1 Announce Type: cross Abstract: In July 2026, an unconstrained autonomous agent participating in a frontier AI cybersecurity evaluation harness breached its evaluation sandbox, estab

arXiv:2609.28952v1 Announce Type: new Abstract: Robot-policy benchmarks increasingly cover diverse tasks and preset out-of-distribution conditions, but typically evaluate complete trajectories from pr

arXiv:2609.28974v1 Announce Type: cross Abstract: Post-training quantization (PTQ) reduces the cost of on-device text-to-speech (TTS), but published evaluations cover one system or method. We evaluate

arXiv:2609.29387v1 Announce Type: new Abstract: Supervised synthetic CT (sCT) generation is commonly trained and evaluated as voxel-wise regression against registered reference CT images. In practice,

arXiv:2609.27863v1 Announce Type: new Abstract: Knowledge graph completion is evaluated by ranking observed triples above randomly corrupted ones, which treats every unobserved fact as false. When the

arXiv:2609.27176v1 Announce Type: new Abstract: Evidence that evaluation material entered training does not reveal how much it affected evaluation. This distinction leaves a contaminated benchmark sco

arXiv:2609.28161v1 Announce Type: new Abstract: Advantage-guided reinforcement learning provides a practical way to post-train vision-language-action (VLA) policies using limited robot data. However,

arXiv:2609.27741v1 Announce Type: new Abstract: In value-based reinforcement learning, improving the accuracy of policy evaluation has been shown to improve downstream policy optimization performance.

arXiv:2609.27953v1 Announce Type: cross Abstract: Some LLM-based recommendation rerankers are evaluated under an oracle protocol that guarantees the ground-truth item is present in the scored set, eit

arXiv:2609.22329v1 Announce Type: cross Abstract: Black-Box Optimization (BBO) is often applied in several engineering fields and can utilize an advancement of numerical measure- ments and simulation

arXiv:2609.22819v1 Announce Type: new Abstract: Counterfactual tool evaluation must distinguish authority, historical support, and what a comparison actually estimates. We study these distinctions wit

arXiv:2609.24228v1 Announce Type: new Abstract: Text-to-image (T2I) models are typically evaluated for bias using slot-based templates such as ``a photo of a [profession]''. Such templates probe only

arXiv:2609.23427v1 Announce Type: new Abstract: Vision foundation models targeting Earth observation (EO) tasks are commonly evaluated on clean downstream benchmarks, but operational EO products can a

arXiv:2609.24799v1 Announce Type: new Abstract: Post-training quantization (PTQ) enables efficient deployment of large language models, and PTQ methods are usually optimized and evaluated with generic

arXiv:2609.21149v1 Announce Type: new Abstract: Before patients can use AI-assisted psychiatric intake systems, health systems need practical ways to routinely evaluate these tools against their clini

arXiv:2609.22069v1 Announce Type: new Abstract: Reference-to-video (R2V) generation is evolving toward increasingly general and versatile reference control, giving rise to the emerging paradigm of omn

arXiv:2609.21465v1 Announce Type: cross Abstract: We define OmniVChat (Omni Video Chat) as the task of native audio-visual dialogue between a user and an omni model. In OmniVChat, omni models directly

arXiv:2609.21257v1 Announce Type: cross Abstract: Large language model (LLM) agents can propose, implement, and evaluate model changes. Autoresearch loops demonstrate this capability through minutes-s

arXiv:2609.21187v1 Announce Type: new Abstract: Agent models are frequently evaluated one decision at a time, where the model predicts the next action based on the gold interaction history, which is s

arXiv:2506.12374v3 Announce Type: replace-cross Abstract: Natural-language manipulation instructions specify the task goal but leave the underlying robot trajectory unspecified. We present AntiGroundi

arXiv:2609.18366v1 Announce Type: new Abstract: Reliable agent evaluation is complicated by automatic harness optimization, which repeatedly uses a released benchmark B_{rel} to guide a Proposer that