Search: “evaluation”

arXiv:2310.00436v2 Announce Type: replace Abstract: Authorship identification uses patterns in writing to infer who wrote a text, but those patterns also reflect topic, genre, and register. This surve

tutorialsarxiv-cs-cl

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

CCTU: A Benchmark for Tool Use under Complex Constraints

DGX agent

arXiv:2603.15309v2 Announce Type: replace Abstract: Solving problems through tool use under explicit constraints constitutes a highly challenging yet unavoidable scenario for large language models (LL

model-releasesarxiv-cs-cl

22 Sept 2026

Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models

model-releasesarxiv-cs-cl

arXiv:2609.24657v1 Announce Type: cross Abstract: Language models can be adapted by changing the computations applied to individual tokens. Quantum circuits offer one such approach, but evaluating wid

Decomposing Error and Style in Automated Clinical Coding

arXiv:2609.24877v1 Announce Type: new Abstract: In automated clinical coding, where the label space spans tens of thousands of diagnosis and procedure codes, models are currently evaluated against a s

Do LiDAR Language Models Really Understand Spatio-temporal Relationships?

arXiv:2609.24452v1 Announce Type: new Abstract: Recent 4D LiDAR language models aim to reason about objects and their evolving spatial relationships. Yet, in our evaluation, always selecting the same

DriveReferee: Geometric Safety Verdicts Need Not Be Learned for Driving World-Action Models

arXiv:2609.22762v1 Announce Type: new Abstract: Generative world-action models (WAMs) jointly generate future video and vehicle actions, while their action branches remain primarily optimized by exper

safetyarxiv-cs-cv

Financial Language Models as Applied Artificial Intelligence Systems for News-Based Trading under Market Frictions

arXiv:2609.23703v1 Announce Type: new Abstract: Financial language models can transform unstructured firm-specific news into structured decision signals, but financial AI research lacks an integrated

researcharxiv-cs-cl

GRACE: Grounded Adversarial Reasoning over Canadian Law

arXiv:2609.23726v1 Announce Type: new Abstract: Large language models have shown strong performance across a range of legal tasks, but existing benchmarks rarely evaluate the ability to take and defen

Learning Physics from an Imperfect Ancestor

arXiv:2609.24947v1 Announce Type: new Abstract: Neural operators evaluate parametric partial differential equations cheaply but degrade sharply outside their training distribution. Physics-informed ne

Leveraging Inference-Time Compute for Diffusion Models via Global Scheduling of Denoising Trajectories

arXiv:2609.22867v1 Announce Type: new Abstract: Diffusion models generate a sample by traversing a denoising trajectory, a sequence of stochastic noise-reduction steps that transforms pure noise into

OSCAR: Order-aware Scoring and Calibration for AI Rankings

arXiv:2609.24128v1 Announce Type: cross Abstract: Judge-specific sensitivity is useful for aggregating pairwise LLM evaluations, but its interpretation depends on which systematic presentation effects

Predictors and Orchestrators: Parsimonious Machine Learning within an Agentic AI Harness for Multi-Horizon Karst Aquifer Forecasting

arXiv:2609.22251v1 Announce Type: new Abstract: Forecasting karst aquifer dynamics is difficult because recharge responses are nonlinear, event-driven, and governed by strongly heterogeneous flow path

Reinforcement learning with an expectile-based objective

arXiv:2602.09300v2 Announce Type: replace Abstract: We consider the policy evaluation and control in a finite horizon reinforcement learning (RL) setting under an expectile-based objective. First, we

Rule-Constrained Assignment for Cue-Ball Identification in Broadcast Snooker

arXiv:2609.23450v1 Announce Type: new Abstract: Accurate cue-ball identification is essential for metric analysis of broadcast snooker. Existing systems evaluate each candidate independently against a

researcharxiv-cs-cv

SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMs

arXiv:2609.22153v1 Announce Type: new Abstract: Methods for addressing safety drift in fine-tuned Large Language Models (LLMs) are scattered across incompatible implementations, lifecycle stages, and

The Operational Value of Spatial Dependence in Renewable Forecast Scenarios for Single-Period Economic Dispatch: A Controlled Ablation Study

arXiv:2609.23988v1 Announce Type: cross Abstract: Renewable forecasts are evaluated by statistical skill (e.g., CRPS), but grid operators pay for realized dispatch cost. We diagnose what drives dispat

researcharxiv-cs-lg

The Situated Identity Test: Distinguishing Persistent Cognitive Identity from Persona Imitation

model-releasesarxiv-cs-cl

arXiv:2609.22195v1 Announce Type: new Abstract: Large language models can convincingly adopt personas, recall past dialogues, and weave rich autobiographies. Yet this conversational eloquence conceals

Token Signatures of Code: Comparing Coding Behaviors Across Large Language Models

arXiv:2609.22097v1 Announce Type: new Abstract: The evaluation of large language models (LLMs) on coding tasks has primarily focused on performance metrics such as pass@k. As LLMs continue to advance,

researcharxiv-cs-cl

Uranus: Building the Next-Generation Simulation Infrastructure for Embodied AI

arXiv:2609.24815v1 Announce Type: new Abstract: Scalable simulation is essential for robot data generation, policy training, evaluation, and safe iteration, yet real-world interaction is costly and co

safetyarxiv-cs-ro

When Residualization Helps an Audit: Format Effects, Slice Gains, and Their Limits

arXiv:2609.24194v1 Announce Type: new Abstract: Evaluation scores used around LLM systems -- including reward models, rerankers, and LLM judges -- can track surface form instead of the quality they cl

X-Beat: An Explainable Framework for ECG Image Classification

arXiv:2609.22631v1 Announce Type: new Abstract: Accurate automated interpretation of electrocardio- grams (ECGs) is essential for early detection of cardiac condi- tions such as myocardial infarction

Co-Evolving Zero-Day Jamming: Adaptive Attack Synthesis and Graph Attention-Based Online Detection

arXiv:2609.21334v1 Announce Type: cross Abstract: Effective evaluation of zero-day jamming detectors requires robust adversarial models. However, existing attack models often assume prior knowledge of

Hardware

How Much of a Real Workload Can LLM-Generated GPU Kernels Actually Reach?

arXiv:2609.21058v1 Announce Type: cross Abstract: Language models can now write GPU kernels that outperform PyTorch. We evaluate five model configurations on KernelBench level 1 and find that a fronti

hardwarearxiv-cs-ai

Implicit Rule Induction with Test-Time Task Embeddings in ARC-like Tasks

arXiv:2609.21181v1 Announce Type: new Abstract: The Abstraction and Reasoning Corpus and related benchmarks evaluate whether AI models can solve novel reasoning tasks, but often leave unclear whether

researcharxiv-cs-ai

PopNavShift: Stress-Testing Social Navigation under Behavioral Population Shift

model-releasesarxiv-cs-ro

arXiv:2609.21838v1 Announce Type: new Abstract: Social-navigation algorithms are often evaluated under a fixed pedestrian-behavior distribution, despite substantial variation in pedestrian responses t

VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration

arXiv:2609.21521v1 Announce Type: cross Abstract: While Video Large Language Models (Video-LLMs) have recently demonstrated strong performance, reliably evaluating their fine-grained video understandi

When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation

arXiv:2609.20942v1 Announce Type: new Abstract: Large language models (LLMs) increasingly participate in scientific evaluation, both as automated reviewers and as assistants to human reviewers. As mod

Local Ai

A Simulation Platform for AUV Fault Recovery: Exploring LLM-Based Diagnostic Strategies

arXiv:2609.20620v1 Announce Type: cross Abstract: Autonomous underwater vehicles (AUVs) operating beyond reliable communications must recover from failures without human intervention. We investigate a

local-aiarxiv-cs-ai

An Empirical Study of Harness Design for Coding Agents

arXiv:2609.20804v1 Announce Type: new Abstract: Coding harnesses shape how autonomous coding agents translate model capabilities into long-horizon software-engineering performance, yet existing work t

AnyviewMeter: Adapting Robotic Reward Models with Camera Geometry and Multi-View Attention

arXiv:2609.20106v1 Announce Type: new Abstract: Robotic reward models evaluate task execution from visual observations, but their predictions can change with camera viewpoint and occlusion even when t

CellRFT: Reinforcement Fine-Tuning for Single-Cell Perturbation Modeling

arXiv:2609.19970v1 Announce Type: new Abstract: Predicting cellular responses to perturbations supports the study of gene function, disease mechanisms, and therapeutic strategies. Despite advances in

Chunk Twice, Embed Once: A Systematic Study of Segmentation and Representation Trade-offs in Chemistry-Aware Retrieval-Augmented Generation

arXiv:2506.17277v2 Announce Type: replace-cross Abstract: The retrieval stage of retrieval-augmented generation (RAG) for scientific question answering depends on how documents are segmented and how c

Agents

EconSkills: Studying Skill Transfer and Retrieval for Web Agents on Live Economic Data

arXiv:2609.19523v1 Announce Type: new Abstract: Web agents often revisit the same sites, yet most evaluations discard the procedures learned in earlier successful interactions. We introduce EconSkills

agentsarxiv-cs-ai

MaSCoD: A Multi-Agent Framework for Structural-Context-Guided Candidate Causal Graph Generation

arXiv:2609.19944v1 Announce Type: new Abstract: Large language models (LLMs) have been applied to causal discovery, but candidate-graph generation rarely treats premature omission of potentially relev

NS3Learn: Transferring 5G NR Mode-2 Reception Realism from ns-3 to the Veins/SUMO Stack for Connected-Vehicle Safety Assessment

arXiv:2609.20578v1 Announce Type: cross Abstract: Connected-vehicle safety evaluations rely on coupled traffic and network simulations, but standard channel models ignore radio resource competition in

Self Improvement via Fast Tree-search

arXiv:2609.19526v1 Announce Type: new Abstract: Coding agents can recursively modify their own implementations, forming a loop of self-improvement. While prior work shows this can boost performance on

SlugTrails: An Egocentric Benchmark for Floor Plan Localization in Large Buildings

arXiv:2609.19876v1 Announce Type: new Abstract: Floor-plan-based indoor visual localization enables infrastructure-free positioning, but most methods are developed and evaluated in small residential e

Agents

What Do We Expect from LLMs? Mapping the Design of LLM Benchmarks

arXiv:2609.19182v1 Announce Type: new Abstract: Benchmarks are central to how progress in large language models (LLMs) is assessed and communicated. Yet model rankings alone reveal little about how ev

agentsarxiv-cs-ai

When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models

arXiv:2603.26556v3 Announce Type: replace-cross Abstract: Converting a pretrained Transformer into a more efficient hybrid model through distillation offers a promising approach to reducing inference

AMIGO: Agentic Multi-Image Grounding Oracle Benchmark

arXiv:2603.28662v2 Announce Type: replace-cross Abstract: Agentic vision-language models increasingly act through extended interactions, but most evaluations still focus on single-image, single-turn c

Local Ai

ASLEval: Measuring Privacy Exposure Displacement in LLM Agent Sessions

arXiv:2609.18864v1 Announce Type: cross Abstract: Privacy evaluations of tool-using LLM agents often inspect a designated action, final response, or attacker report. These local proxies can miss unaut

local-aiarxiv-cs-ai

Benchmarking Visual-Inertial Odometry in Subterranean Environments Under Sensor Degradation, Miscalibration, and Dynamic Occlusion

model-releasesarxiv-cs-ro

arXiv:2609.18628v1 Announce Type: new Abstract: Visual-inertial odometry (VIO) is a core capability for autonomous operation in GPS-denied subterranean environments, yet its reliability can degrade sh

Cultural Competence in Context: A Large Language Model Passes the Turing Test in Finland

arXiv:2609.18394v1 Announce Type: new Abstract: We report the results of a Turing Test conducted in Finland in the Finnish language. Because languages and cultural contexts are unevenly represented in

researcharxiv-cs-ai

CzechTopic: A Benchmark for Zero-Shot Topic Localization in Historical Czech Documents

arXiv:2603.03884v2 Announce Type: replace-cross Abstract: Topic localization aims to identify spans of text that express a given topic defined by a name and description. To study this task, we introdu

Faking Good and Faking Bad in LLMs: Response Distortion Across Dark Triad Personality Traits

arXiv:2609.17534v1 Announce Type: new Abstract: Social desirability and impression management are pervasive sources of response distortion in human personality assessment, yet their effects on Large L

Agents

Market Signal Injection: Adversarial Context Manipulation of LLM Pricing Agents

arXiv:2609.18357v1 Announce Type: new Abstract: Large language model (LLM) pricing agents may respond to how market data is presented, even when its numerical values remain unchanged. We introduce mar

agentsarxiv-cs-ai

Prosthesis-Aware 3D Human Pose Estimation: A Dataset and Benchmark for RSP Users

arXiv:2609.18406v1 Announce Type: new Abstract: Recovering 3D human body motion from video is important for applications such as rehabilitation assessment and sports performance evaluation. For prosth

Reading Between the Lines: Can LLMs Discover the Question Behind the Text?

model-releasesarxiv-cs-cl

arXiv:2609.19070v1 Announce Type: new Abstract: This paper introduces ``question archaeology'', a specific evaluation task focused on inferring the single, authentic 'genesis question' that motivated