Search: “evaluation”

Scaling Versatile 3D Assets Editing with a Million-Scale Dataset

model-releasesarxiv-cs-cv

arXiv:2609.34271v1 Announce Type: new Abstract: Although recent 3D generative models produce increasingly realistic assets, controllable 3D asset editing remains challenging. Existing methods are limi

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

SPACE: Sparse Predictive Attractor via Counterfactual Eviction for Streaming Video Memory

DGX agent

arXiv:2609.32592v1 Announce Type: new Abstract: Fixed-capacity streaming video memory requires repeated eviction decisions whose effects accumulate over time. Yet existing policies are evaluated prima

model-releasesarxiv-cs-cv

29 Sept 2026

The Composition Gap in Dataset Distillation

arXiv:2609.34343v1 Announce Type: new Abstract: Dataset distillation compresses a training set into a small synthetic set, usually evaluated one at a time. In federated and data-governance settings, s

The Price of Peeking: Anytime-Valid Leakage Detection on ML-KEM EM Traces

arXiv:2609.33597v1 Announce Type: cross Abstract: Side-channel evaluators routinely inspect leakage tests while acquisition is still running, and extend or stop the campaign based on what they see. Fi

Verifying the Linear Representation Hypothesis: How Interpretable Are Vision SAEs?

arXiv:2609.35020v1 Announce Type: new Abstract: Vision Sparse Autoencoders (SAEs) have become a popular tool in Mechanistic Interpretability due to their presumed ability to disentangle complex featur

safetyarxiv-cs-cv

VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation

arXiv:2609.35028v1 Announce Type: new Abstract: Large language models (LLMs) have made misinformation inexpensive to produce but not to verify, creating a growing asymmetry in the information ecosyste

Who Gets a Token, and What Does It Carry? Unequal Name Support and Concept Access in Large Language Models

arXiv:2609.34065v1 Announce Type: cross Abstract: Names are personal identifiers, but they also carry social meaning and are widely used to evaluate how language models treat different people. Such ev

Word Similarity Datasets for Indian Languages: Annotation and Baseline Systems

arXiv:2609.34138v1 Announce Type: cross Abstract: With the advent of word representations, word similarity tasks are becoming increasing popular as an evaluation metric for the quality of the represen

ZeroGAR: Benchmarking the Adversarial Robustness of Zero-Shot Graph Models

arXiv:2609.33314v1 Announce Type: new Abstract: Zero-shot graph models (ZGMs), which learn transferable knowledge from source graphs and directly apply to unseen target graphs without any adaptation,

A Benchmark Framework for Screening Automation in Systematic Reviews

arXiv:2609.30298v1 Announce Type: new Abstract: Systematic reviews (SR) are essential for evidence-based research, but their screening phase is highly time-consuming and labor-intensive. Large languag

AcoustiClaim: A Numeric Claim Benchmark with Instrument Ground Truth

arXiv:2609.30483v1 Announce Type: cross Abstract: Audio language models state numbers for acoustic quantities, and neither human opinion nor a judge model says whether such a number is true of the sig

Aligning One-Step Generative Models with Reward-Weighted Transport Distillation

arXiv:2609.30840v1 Announce Type: cross Abstract: One-step generators enable high-quality visual generation with a single network evaluation, but their post-training is difficult: general implicit gen

safetyarxiv-cs-cv

Beyond Empirical Support: Structured Outlier Generation via Sinkhorn Optimal Transport

arXiv:2609.31470v1 Announce Type: cross Abstract: Outliers are essential for evaluating and improving the robustness of machine learning systems, especially when future distributions may differ signif

Compact Documentation for Coding Agents: A Benchmark, an Optimizer, and Why It Does Not Transfer

arXiv:2609.31587v1 Announce Type: cross Abstract: We investigate whether natural-language documentation helps coding agents resolve software issues, and we build the tools to construct and evaluate it

DepthEvidence: Unifying Metric Depth Prediction and Geometric Reasoning in Multimodal Language Models

arXiv:2609.31103v1 Announce Type: cross Abstract: Spatial reasoning with metric constraints requires linking objects to geometric measurements and preserving their numerical content during language re

DIAL: Position-Debiased LLM Judges with Adaptive Human Preference Calibration

arXiv:2609.31215v1 Announce Type: new Abstract: Large language models (LLMs) as a judge enable scalable evaluation, but their judgments can be sensitive to response order and, even after removing such

safetyarxiv-cs-ai

LiTe-GS: Oracle-Efficient Next Best View Selection for 3D Gaussian Splatting

arXiv:2609.30393v1 Announce Type: new Abstract: Selecting informative camera views is critical for efficient training and adaptive refinement in 3D Gaussian Splatting, where each observation significa

researcharxiv-cs-cv

Probing Stability-Plasticity Tradeoffs in Agent Memory through Cognitive Experimental Paradigms

arXiv:2609.30558v1 Announce Type: cross Abstract: Agent memory systems are increasingly used to maintain long-term user preferences, task states and evolving facts, but current evaluations often colla

Same Text, Different Numbers: The Divergence of LLM-Based Measures

arXiv:2609.31013v1 Announce Type: new Abstract: Researchers increasingly use generative large language models (LLMs) to convert corporate text into empirical variables. We examine the extent to which

SciHorizon-eLab: An Agentic Protocol-to-Task Compiler for Scalable Benchmarking of Scientific Embodied Agents

arXiv:2609.30971v1 Announce Type: new Abstract: Embodied agents offer a promising route to automating scientific experimentation, yet their progress is constrained by the lack of reliable and systemat

Towards Understanding LLM-Based Log Anomaly Detection: An Empirical Study of Performance, Efficiency, and Robustness

arXiv:2609.31371v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated promising performance in log anomaly detection, yet how their adaptation strategies, architectures, and d

When 10,000 Windows Are Not 10,000 Tests: Auditing Statistical Confidence in Sliding-Window Time-Series Classification

arXiv:2609.30721v1 Announce Type: new Abstract: Sliding-window classifiers are often evaluated on thousands of overlapping test windows, even though neighboring predictions share observations and rema

XPhysICS: Cross-Physical-Domain Threat Grounding for Industrial Control Systems Security

arXiv:2609.30805v1 Announce Type: cross Abstract: Industrial control system (ICS) threats documented for one plant can express cyber-physical effects relevant to another, but semantic similarity alone

A Living Benchmark for Information Retrieval from Electronic Health Records

arXiv:2609.30205v1 Announce Type: new Abstract: Large language model (LLM)-based clinical assistants are increasingly being integrated into electronic health record (EHR) systems, transforming how cli

Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems

arXiv:2609.30001v1 Announce Type: new Abstract: Sustaining industrial recommendation research requires using the results of one experiment to decide what to investigate next. We present AgentX-Model,

Beyond Surface Style: Aligning Multi-Turn User Simulators with Behavioral Consistency

arXiv:2609.28690v1 Announce Type: new Abstract: Faithful user simulation is fundamental to building, evaluating, and improving interactive AI at scale. However, plausible individual responses do not e

Combining Evasive and Braking Reactions for Safety Reference Models in Automated Vehicles

arXiv:2609.29738v1 Announce Type: new Abstract: Computational models of careful and competent human drivers are essential for scenario-based evaluation of automated driving systems (ADS). However, mos

safetyarxiv-cs-ro

DiscoPhon: Benchmarking the Unsupervised Discovery of Phoneme Inventories With Discrete Speech Units

arXiv:2603.18612v2 Announce Type: replace Abstract: We introduce DiscoPhon, a multilingual benchmark for evaluating unsupervised phoneme discovery from discrete speech units. DiscoPhon covers 6 dev an

Does DCGAN-Based Synthetic Augmentation Improve Brain Tumor MRI Classification? An Empirical Study

arXiv:2609.28508v1 Announce Type: cross Abstract: Generative adversarial networks (GANs) are increasingly used to augment medical imaging datasets, but synthetic images do not necessarily provide down

researcharxiv-cs-cv

Forecast-Dojo: Replayable Environments for Benchmarking and Training LLM Forecasting Agents

arXiv:2609.28876v1 Announce Type: new Abstract: We introduce Forecast-Dojo, a replayable environment for benchmarking and training LLM forecasting agents. It combines resolved prediction-market questi

Human-AI-Powered Hypothesis Testing: Cost-Aware Selective AI Scoring and Sequential Human Escalation

arXiv:2609.28859v1 Announce Type: new Abstract: Large language models are increasingly used as inexpensive judges to evaluate outputs, label data, and assess whether a system meets a desired quality s

safetyarxiv-cs-ai

Learning to Ideate for Scientific Impact

arXiv:2609.29802v1 Announce Type: new Abstract: Scientific ideation is increasingly mediated by large language models, but current ideation systems are usually trained and evaluated on immediately jud

Reasoning Instructions Can Break Answer Decoding in Vision--Language Models

arXiv:2609.29278v1 Announce Type: cross Abstract: Chain-of-thought (CoT) instructions can distort multiple-choice VLM evaluation when a scorer appends a reasoning cue but reads answer-label logits bef

Sequential knowledge editing breaks a model's ability to tell good evidence from bad, without costing it accuracy

arXiv:2609.29587v1 Announce Type: new Abstract: Knowledge editing is evaluated on whether the edited fact changed, whether paraphrases follow, and whether unrelated answers stayed put. A model can pas

StepCOPS: Closed-Testing Lower-Tail Certificates for Language-Model Policy Selection

arXiv:2609.29549v1 Announce Type: new Abstract: Post-training pipelines must select one language-model policy from many checkpoints, prompts, and decoding rules. Mean evaluator scores can conceal rare

SWE-Prometheus: Measuring Engineering Governance Improvements in Real-World Repositories

arXiv:2609.29465v1 Announce Type: new Abstract: Large language model based coding agents have made substantial progress on repository-level software engineering tasks. Existing repository benchmarks,

The Fellowship of the Query: Learning Retrieval Actions

arXiv:2609.28653v1 Announce Type: cross Abstract: Retrieval-augmented question answering requires control decisions about when to decompose a question, search, reformulate, extract evidence, synthesiz

Three Ways Classical Test Theory Misleads for LLM Judges

arXiv:2609.29709v1 Announce Type: cross Abstract: An LLM judge scores a bank of responses against a rubric, and the reliability comes back at 0.52. What has been measured? Judge evaluation has begun b

researcharxiv-cs-cl

Who Holds the Pen? Let Specifications, Not Agents, Sign Off

arXiv:2609.29921v1 Announce Type: new Abstract: Large language model agents increasingly combine generation, decision-making, execution, and self-evaluation within a single agentic loop. Although they

Building Socio-Affective Artificial Intelligence for Interactive Multi-Agent Simulations

arXiv:2609.26927v1 Announce Type: new Abstract: The objective of this article is to provide design principles and a software architecture for enabling interaction between humans and multiple agents in

Can LLMs Reason About Runtime Behavior? A Repository-Level Dynamic Benchmark

arXiv:2609.28449v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in coding tasks, but their ability to reason about code execution remains unclear. Existing reposit

Classifying Interpretive Canons at the Sentence Level: A Benchmark from the German Federal Constitutional Court

arXiv:2609.26945v1 Announce Type: new Abstract: Judicial reasoning remains challenging for large language models (LLMs) to analyze. This paper contributes a sentence-level benchmark for evaluating the

Compressed delayed-information projection for six-degree-of-freedom underwater vehicle navigation under delayed acoustic positioning

arXiv:2609.27439v1 Announce Type: new Abstract: Delayed acoustic positioning packets constrain historical navigation states, but a current-time update evaluates them against a mismatched state, wherea

researcharxiv-cs-ro

EvEMTBench: An Open Benchmark for Machine Learning in Power System Protection

arXiv:2609.28149v1 Announce Type: cross Abstract: Studies of machine-learning-based power system protection are difficult to compare because task definitions, measurement access, data partitions, metr

KATOsuper: Surrogate-accelerated neural topology optimization with sensitivity-consistent Fourier neural operators

arXiv:2609.27216v1 Announce Type: cross Abstract: Topology optimization (TO) remains computationally intensive due to repeated finite element analysis (FEA) evaluations required at each iteration. Whi

Local Ai

LAYERSCOPE: A Layerwise Characterization of Video and Multimodal Learned Representations

arXiv:2609.28086v1 Announce Type: cross Abstract: We propose LAYERSCOPE, a label-free, layerwise framework that aims to characterize a model's learned representations in video and multimodal settings.

local-aiarxiv-cs-ai

MWE-ECL: Recoverable Long-Range Context Does Not Always Override Local Lexical Priors

arXiv:2609.27590v1 Announce Type: new Abstract: Long-context evaluations often test whether a model can recover distant evidence, but recoverability does not guarantee behavioral influence. We test th

Provably Complete Generalized Planning with LLMs

arXiv:2609.27105v1 Announce Type: new Abstract: Generalized planning aims to compute a plan that solves all instances of a planning domain. Recent work has used LLMs to automatically generate and debu