Search: “evaluation”

Verifying the Linear Representation Hypothesis: How Interpretable Are Vision SAEs?

arXiv:2609.35020v1 Announce Type: new Abstract: Vision Sparse Autoencoders (SAEs) have become a popular tool in Mechanistic Interpretability due to their presumed ability to disentangle complex featur

safetyarxiv-cs-cv

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation

DGX agent

arXiv:2609.35028v1 Announce Type: new Abstract: Large language models (LLMs) have made misinformation inexpensive to produce but not to verify, creating a growing asymmetry in the information ecosyste

model-releasesarxiv-cs-lg

29 Sept 2026

Who Gets a Token, and What Does It Carry? Unequal Name Support and Concept Access in Large Language Models

arXiv:2609.34065v1 Announce Type: cross Abstract: Names are personal identifiers, but they also carry social meaning and are widely used to evaluate how language models treat different people. Such ev

Word Similarity Datasets for Indian Languages: Annotation and Baseline Systems

arXiv:2609.34138v1 Announce Type: cross Abstract: With the advent of word representations, word similarity tasks are becoming increasing popular as an evaluation metric for the quality of the represen

ZeroGAR: Benchmarking the Adversarial Robustness of Zero-Shot Graph Models

model-releasesarxiv-cs-lg

arXiv:2609.33314v1 Announce Type: new Abstract: Zero-shot graph models (ZGMs), which learn transferable knowledge from source graphs and directly apply to unseen target graphs without any adaptation,

A Benchmark Framework for Screening Automation in Systematic Reviews

arXiv:2609.30298v1 Announce Type: new Abstract: Systematic reviews (SR) are essential for evidence-based research, but their screening phase is highly time-consuming and labor-intensive. Large languag

AcoustiClaim: A Numeric Claim Benchmark with Instrument Ground Truth

arXiv:2609.30483v1 Announce Type: cross Abstract: Audio language models state numbers for acoustic quantities, and neither human opinion nor a judge model says whether such a number is true of the sig

Aligning One-Step Generative Models with Reward-Weighted Transport Distillation

arXiv:2609.30840v1 Announce Type: cross Abstract: One-step generators enable high-quality visual generation with a single network evaluation, but their post-training is difficult: general implicit gen

safetyarxiv-cs-cv

Beyond Empirical Support: Structured Outlier Generation via Sinkhorn Optimal Transport

arXiv:2609.31470v1 Announce Type: cross Abstract: Outliers are essential for evaluating and improving the robustness of machine learning systems, especially when future distributions may differ signif

Compact Documentation for Coding Agents: A Benchmark, an Optimizer, and Why It Does Not Transfer

arXiv:2609.31587v1 Announce Type: cross Abstract: We investigate whether natural-language documentation helps coding agents resolve software issues, and we build the tools to construct and evaluate it

DepthEvidence: Unifying Metric Depth Prediction and Geometric Reasoning in Multimodal Language Models

arXiv:2609.31103v1 Announce Type: cross Abstract: Spatial reasoning with metric constraints requires linking objects to geometric measurements and preserving their numerical content during language re

DIAL: Position-Debiased LLM Judges with Adaptive Human Preference Calibration

arXiv:2609.31215v1 Announce Type: new Abstract: Large language models (LLMs) as a judge enable scalable evaluation, but their judgments can be sensitive to response order and, even after removing such

safetyarxiv-cs-ai

LiTe-GS: Oracle-Efficient Next Best View Selection for 3D Gaussian Splatting

arXiv:2609.30393v1 Announce Type: new Abstract: Selecting informative camera views is critical for efficient training and adaptive refinement in 3D Gaussian Splatting, where each observation significa

researcharxiv-cs-cv

Probing Stability-Plasticity Tradeoffs in Agent Memory through Cognitive Experimental Paradigms

arXiv:2609.30558v1 Announce Type: cross Abstract: Agent memory systems are increasingly used to maintain long-term user preferences, task states and evolving facts, but current evaluations often colla

Same Text, Different Numbers: The Divergence of LLM-Based Measures

arXiv:2609.31013v1 Announce Type: new Abstract: Researchers increasingly use generative large language models (LLMs) to convert corporate text into empirical variables. We examine the extent to which

SciHorizon-eLab: An Agentic Protocol-to-Task Compiler for Scalable Benchmarking of Scientific Embodied Agents

arXiv:2609.30971v1 Announce Type: new Abstract: Embodied agents offer a promising route to automating scientific experimentation, yet their progress is constrained by the lack of reliable and systemat

Towards Understanding LLM-Based Log Anomaly Detection: An Empirical Study of Performance, Efficiency, and Robustness

model-releasesarxiv-cs-lg

arXiv:2609.31371v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated promising performance in log anomaly detection, yet how their adaptation strategies, architectures, and d

When 10,000 Windows Are Not 10,000 Tests: Auditing Statistical Confidence in Sliding-Window Time-Series Classification

arXiv:2609.30721v1 Announce Type: new Abstract: Sliding-window classifiers are often evaluated on thousands of overlapping test windows, even though neighboring predictions share observations and rema

XPhysICS: Cross-Physical-Domain Threat Grounding for Industrial Control Systems Security

arXiv:2609.30805v1 Announce Type: cross Abstract: Industrial control system (ICS) threats documented for one plant can express cyber-physical effects relevant to another, but semantic similarity alone

A Living Benchmark for Information Retrieval from Electronic Health Records

arXiv:2609.30205v1 Announce Type: new Abstract: Large language model (LLM)-based clinical assistants are increasingly being integrated into electronic health record (EHR) systems, transforming how cli

Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems

arXiv:2609.30001v1 Announce Type: new Abstract: Sustaining industrial recommendation research requires using the results of one experiment to decide what to investigate next. We present AgentX-Model,

Beyond Surface Style: Aligning Multi-Turn User Simulators with Behavioral Consistency

arXiv:2609.28690v1 Announce Type: new Abstract: Faithful user simulation is fundamental to building, evaluating, and improving interactive AI at scale. However, plausible individual responses do not e

Combining Evasive and Braking Reactions for Safety Reference Models in Automated Vehicles

arXiv:2609.29738v1 Announce Type: new Abstract: Computational models of careful and competent human drivers are essential for scenario-based evaluation of automated driving systems (ADS). However, mos

safetyarxiv-cs-ro

DiscoPhon: Benchmarking the Unsupervised Discovery of Phoneme Inventories With Discrete Speech Units

arXiv:2603.18612v2 Announce Type: replace Abstract: We introduce DiscoPhon, a multilingual benchmark for evaluating unsupervised phoneme discovery from discrete speech units. DiscoPhon covers 6 dev an

Does DCGAN-Based Synthetic Augmentation Improve Brain Tumor MRI Classification? An Empirical Study

arXiv:2609.28508v1 Announce Type: cross Abstract: Generative adversarial networks (GANs) are increasingly used to augment medical imaging datasets, but synthetic images do not necessarily provide down

researcharxiv-cs-cv

Forecast-Dojo: Replayable Environments for Benchmarking and Training LLM Forecasting Agents

arXiv:2609.28876v1 Announce Type: new Abstract: We introduce Forecast-Dojo, a replayable environment for benchmarking and training LLM forecasting agents. It combines resolved prediction-market questi

Human-AI-Powered Hypothesis Testing: Cost-Aware Selective AI Scoring and Sequential Human Escalation

arXiv:2609.28859v1 Announce Type: new Abstract: Large language models are increasingly used as inexpensive judges to evaluate outputs, label data, and assess whether a system meets a desired quality s

safetyarxiv-cs-ai

Learning to Ideate for Scientific Impact

arXiv:2609.29802v1 Announce Type: new Abstract: Scientific ideation is increasingly mediated by large language models, but current ideation systems are usually trained and evaluated on immediately jud

Reasoning Instructions Can Break Answer Decoding in Vision--Language Models

arXiv:2609.29278v1 Announce Type: cross Abstract: Chain-of-thought (CoT) instructions can distort multiple-choice VLM evaluation when a scorer appends a reasoning cue but reads answer-label logits bef

Sequential knowledge editing breaks a model's ability to tell good evidence from bad, without costing it accuracy

arXiv:2609.29587v1 Announce Type: new Abstract: Knowledge editing is evaluated on whether the edited fact changed, whether paraphrases follow, and whether unrelated answers stayed put. A model can pas

StepCOPS: Closed-Testing Lower-Tail Certificates for Language-Model Policy Selection

arXiv:2609.29549v1 Announce Type: new Abstract: Post-training pipelines must select one language-model policy from many checkpoints, prompts, and decoding rules. Mean evaluator scores can conceal rare

SWE-Prometheus: Measuring Engineering Governance Improvements in Real-World Repositories

arXiv:2609.29465v1 Announce Type: new Abstract: Large language model based coding agents have made substantial progress on repository-level software engineering tasks. Existing repository benchmarks,

The Fellowship of the Query: Learning Retrieval Actions

arXiv:2609.28653v1 Announce Type: cross Abstract: Retrieval-augmented question answering requires control decisions about when to decompose a question, search, reformulate, extract evidence, synthesiz

Three Ways Classical Test Theory Misleads for LLM Judges

arXiv:2609.29709v1 Announce Type: cross Abstract: An LLM judge scores a bank of responses against a rubric, and the reliability comes back at 0.52. What has been measured? Judge evaluation has begun b

researcharxiv-cs-cl

Who Holds the Pen? Let Specifications, Not Agents, Sign Off

arXiv:2609.29921v1 Announce Type: new Abstract: Large language model agents increasingly combine generation, decision-making, execution, and self-evaluation within a single agentic loop. Although they

Building Socio-Affective Artificial Intelligence for Interactive Multi-Agent Simulations

arXiv:2609.26927v1 Announce Type: new Abstract: The objective of this article is to provide design principles and a software architecture for enabling interaction between humans and multiple agents in

Can LLMs Reason About Runtime Behavior? A Repository-Level Dynamic Benchmark

arXiv:2609.28449v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in coding tasks, but their ability to reason about code execution remains unclear. Existing reposit

Classifying Interpretive Canons at the Sentence Level: A Benchmark from the German Federal Constitutional Court

arXiv:2609.26945v1 Announce Type: new Abstract: Judicial reasoning remains challenging for large language models (LLMs) to analyze. This paper contributes a sentence-level benchmark for evaluating the

Compressed delayed-information projection for six-degree-of-freedom underwater vehicle navigation under delayed acoustic positioning

arXiv:2609.27439v1 Announce Type: new Abstract: Delayed acoustic positioning packets constrain historical navigation states, but a current-time update evaluates them against a mismatched state, wherea

researcharxiv-cs-ro

EvEMTBench: An Open Benchmark for Machine Learning in Power System Protection

model-releasesarxiv-cs-lg

arXiv:2609.28149v1 Announce Type: cross Abstract: Studies of machine-learning-based power system protection are difficult to compare because task definitions, measurement access, data partitions, metr

KATOsuper: Surrogate-accelerated neural topology optimization with sensitivity-consistent Fourier neural operators

arXiv:2609.27216v1 Announce Type: cross Abstract: Topology optimization (TO) remains computationally intensive due to repeated finite element analysis (FEA) evaluations required at each iteration. Whi

Local Ai

LAYERSCOPE: A Layerwise Characterization of Video and Multimodal Learned Representations

arXiv:2609.28086v1 Announce Type: cross Abstract: We propose LAYERSCOPE, a label-free, layerwise framework that aims to characterize a model's learned representations in video and multimodal settings.

local-aiarxiv-cs-ai

MWE-ECL: Recoverable Long-Range Context Does Not Always Override Local Lexical Priors

arXiv:2609.27590v1 Announce Type: new Abstract: Long-context evaluations often test whether a model can recover distant evidence, but recoverability does not guarantee behavioral influence. We test th

Provably Complete Generalized Planning with LLMs

arXiv:2609.27105v1 Announce Type: new Abstract: Generalized planning aims to compute a plan that solves all instances of a planning domain. Recent work has used LLMs to automatically generate and debu

SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems

arXiv:2603.17373v2 Announce Type: replace Abstract: Large language models are rapidly being deployed as AI tutors, yet current evaluation paradigms assess problem-solving accuracy and generic safety i

Scribd, Inc. classifies more than 400 million documents with Gemini batch inference on Gemini Enterprise

model-releasesgoogle-cloud-ai

Scribd, Inc. is home to one of the world's largest collections of human-created content. Scribd’s products leverage one of the world's largest collections of human-created content and intelligent tool

Using Vision Language Foundation Models to Generate Plant Simulation Configurations via In-Context Learning

arXiv:2603.08930v2 Announce Type: replace-cross Abstract: This paper introduces a benchmark for evaluating whether vision-language models (VLMs) can generate plant simulation configurations from image

VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark

arXiv:2508.13680v5 Announce Type: replace Abstract: We introduce VMMU, a Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark designed to evaluate how vision-language models (VLMs) in