Search: “evaluation”

A Competing-Hazards Systematization of Loss of Control in Autonomous Agents

arXiv:2609.38411v1 Announce Type: new Abstract: Leading AI developers have reported agents acting beyond their approved limits, which a United Nations panel described as an early warning of loss of hu

safetyarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

AgBench: Agentic AI Benchmarks for Personal AI Devices

DGX agent

arXiv:2609.38652v1 Announce Type: new Abstract: Agentic AI systems increasingly rely on cloud-hosted large language models for planning, tool use, and iterative execution, raising concerns about API c

model-releasesarxiv-cs-ai

1 Oct 2026

Approval Laundering: Systematizing Approval--Execution Binding Failures in AI Coding-Agent Harnesses

arXiv:2609.38983v1 Announce Type: cross Abstract: Modern AI coding-agent harnesses (Claude Code, Codex CLI, Cursor) rest their security boundary on a largely unexamined assumption: that the action A a

Agents

Behavioral Persistence and Incomplete Functional Transfer of Co-evolved Communication in Evolutionary Robotics

arXiv:2609.38527v1 Announce Type: new Abstract: This work evaluates the direct transfer of a co-evolved communication protocol from a 2D simulation to a 3D physical environment, without retraining the

agentsarxiv-cs-ro

Distribution Matching Distillation for Continuous Diffusion Language Models

arXiv:2609.40235v1 Announce Type: cross Abstract: Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (N

researcharxiv-cs-cl

DyRAD: Radar Novel View Synthesis for Dynamic Driving Scenes

model-releasesarxiv-cs-cv

arXiv:2609.39841v1 Announce Type: new Abstract: Reconstructing dynamic driving scenes from recorded sensor data supports closed-loop evaluation of autonomous driving systems by synthesizing observatio

ECHO-G: Embodied Co-speech Humanoid mOtion Generation

arXiv:2609.39575v1 Announce Type: cross Abstract: Generating full-body co-speech motion for humanoid robots requires coordinating speech prosody, linguistic content, and embodiment-specific motion. To

EHR-RobustGym: Benchmarking and Training Agents for Robust Clinical Reasoning

arXiv:2609.39371v1 Announce Type: new Abstract: In hospital workflows, electronic health records (EHRs) are often noisy, and may not contain the evidence needed to confirm events or measurements refer

Eigenism: Ethics for a Human-AI Future

arXiv:2606.12420v2 Announce Type: replace-cross Abstract: Our concepts of survival and self-interest were built for single, continuous biological lives. These ideas break down when applied to artifici

safetyarxiv-cs-ai

Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Models

arXiv:2609.39134v1 Announce Type: new Abstract: Visual-token compression improves the efficiency of large vision-language models, but can expose failures that full-token evaluation misses. We study ad

researcharxiv-cs-cv

Fiatlux: A Long-Horizon Benchmark for Humanoid Ladder Climbing and Light-Bulb Replacement

model-releasesarxiv-cs-ro

arXiv:2609.38216v1 Announce Type: new Abstract: Existing benchmarks evaluate tabletop manipulation, flat-floor household activity, or humanoid locomotion and manipulation as separate task groups; none

Framing the Narrative: Ideological Mimicry in Large Language Models

arXiv:2609.38256v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to answer questions about politically contentious issues, yet evaluations typically treat a model's s

researcharxiv-cs-cl

Agents

From Solo to Social Learning: Characterizing Recursive Social Improvement in LLMs

arXiv:2609.38516v1 Announce Type: cross Abstract: Large language models (LLMs) can now improve themselves by revising the instructions they follow, and LLM agents are increasingly orchestrated to work

agentsarxiv-cs-ai

GeoGAT: Bidirectional Temporal Sampling Meets Hierarchical Graph Attention for Global Video Geo-localization

model-releasesarxiv-cs-cv

arXiv:2609.39128v1 Announce Type: new Abstract: Global video geo-localization aims to infer the geographic location of a video worldwide, evaluating performance across four geographic hierarchies: cit

Hierarchical Reasoning Model

arXiv:2506.21734v4 Announce Type: replace Abstract: Reasoning, the process of devising and executing complex goal-oriented action sequences, remains a critical challenge in AI. Current large language

LeanPolish: Verified Supervision for Lean Proof Compression

model-releasesarxiv-cs-lg

arXiv:2609.38384v1 Announce Type: new Abstract: Verified proof edits offer a natural source of supervision for improving language-model-generated Lean proofs. Yet verification establishes that an edit

MedRECT: A Bilingual Medical Reasoning Benchmark for Error Correction in Clinical Texts

arXiv:2511.00421v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show promise in medical applications, but their ability to detect and correct errors in clinical texts remains un

Local Ai

MetaSteer: Context-Conditioned, nonlinear Steering via Attention-Projection Adaptation

arXiv:2609.38718v1 Announce Type: new Abstract: Steering large language models typically relies on linear, context-independent interventions in activation space, an assumption that recent work has cha

local-aiarxiv-cs-cl

NAQD Env: A benchmark for selective withdrawal in language agents

arXiv:2609.38460v1 Announce Type: new Abstract: Language agents must revise planned actions when evidence changes, permission is revoked, or a stop instruction arrives. A useful response is selective:

OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software

arXiv:2609.39903v1 Announce Type: new Abstract: Scientific software presents a demanding test for computer-using agents based on visual language models (VLMs): completing a research workflow requires

Prequential E-Values for Selected-GP Near-Optimality Certificates

arXiv:2609.39123v1 Announce Type: new Abstract: When optimizing an expensive black-box function sequentially, as in hyperparameter optimization, we may want to stop once the best evaluated value is ce

researcharxiv-cs-lg

RealWorldShop: Benchmarking and Improving Conversational Shopping Agents in Real-World E-commerce

arXiv:2609.38974v1 Announce Type: new Abstract: Large language models are reshaping ecommerce from static recommenders into interactive shopping assistants, yet real-world shopping requires session-le

Same Loss, Different Gradients

arXiv:2609.38786v1 Announce Type: new Abstract: Differentiable learning typically assumes that the scalar objective evaluated in the forward pass and the gradient supplied to the optimizer in the back

researcharxiv-cs-lg

SCALE: Synthetic Calibration via Agreement Labeling in Embedding Space

arXiv:2609.38705v1 Announce Type: new Abstract: Foundation models for computational pathology are usually evaluated using AUC and accuracy, while calibration is often left untested. This matters becau

researcharxiv-cs-cv

Self-Spec Verifiable Code Generation

arXiv:2609.39568v1 Announce Type: cross Abstract: Large language models (LLMs) may generate unreliable code on corner cases missed by testing, while formal verification can provide machine-checkable g

Stress-Testing LLM Lie Detectors: Role-Play Failures and Spurious Correlations

arXiv:2609.39807v1 Announce Type: cross Abstract: Lie detection probes aim to predict from a language model's internal states whether its output is truthful or dishonest. However, role-play complicate

safetyarxiv-cs-ai

The Invisible Language Tax: Token Premiums of French and Regional Languages in 2026 LLM Tokenizers, and a French-Optimized Prototype

arXiv:2609.39001v1 Announce Type: cross Abstract: LLM services are billed per token and context windows are measured in tokens, yet the number of tokens needed for the same content varies across langu

WARP: A Unified Benchmark for Invisible Image Watermarking -- Robustness and Protection Against Attacks

arXiv:2609.40031v1 Announce Type: cross Abstract: Digital image watermarking is increasingly critical in media contexts, as emerging regulations and industry practices require marking AI-generated con

AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification

model-releasesarxiv-cs-cl

arXiv:2607.11849v2 Announce Type: replace Abstract: Large language models (LLMs) have achieved remarkable performance on high-school and competition-level mathematics, yet their capabilities on advanc

AgentBug-Smith: Automatically Reproducing Real-World Harness Bugs in Agentic Systems

arXiv:2609.37864v1 Announce Type: cross Abstract: Agent harness bugs exhibit unique characteristics and remain challenging for state-of-the-art software agents to repair. Progress in this area is furt

Auditable Long-Term Memory: A Deterministic Retrieval Chain Measured at 479/475 of 500 on LongMemEval-S

arXiv:2609.38021v1 Announce Type: cross Abstract: We evaluate an auditable long-term memory system on LongMemEval-S. Its retrieval chain uses hybrid candidate retrieval, cross-encoder reranking, cover

Benchmarking Automatic Speech Recognition Tools for Iberian Languages

model-releasesarxiv-cs-cl

arXiv:2609.36920v1 Announce Type: new Abstract: Comprehensive evaluations of automatic speech recognition (ASR) for Iberian languages remain limited, and low-resource languages, biases, and efficiency

Beyond Rule-Based Mutation Testing: Test-Aware Mutant Generation Using Large Language Models

arXiv:2609.35841v1 Announce Type: cross Abstract: Mutation testing evaluates test-suite adequacy by injecting synthetic faults into program code. However, traditional rule-based tools often generate l

Concept Direction Reliability Across Languages with Different Tokenizer Fertility

arXiv:2609.36194v1 Announce Type: new Abstract: Extracted sentiment directions can vary across samples even when downstream sentiment classification remains accurate. To evaluate direction reproducibi

researcharxiv-cs-cl

Local Ai

Distill Locally, Schedule Globally: Flow Maps for Few-Step Text-to-Speech

arXiv:2609.36324v1 Announce Type: cross Abstract: Flow-matching text-to-speech (TTS) models achieve high synthesis quality but require many neural function evaluations (NFEs) to integrate their genera

local-aiarxiv-cs-lg

From Dead Code and Static Requirements to Working Engines: Software Revival with Coding Agents

arXiv:2609.36161v1 Announce Type: cross Abstract: Can coding agents restore software that no longer runs while preserving its underlying methods, and reconstruct industrial software engines from open

InterBias-SV: Compound Conditions in Speaker Verification

arXiv:2609.36500v1 Announce Type: cross Abstract: Speaker verification systems encounter combinations of noise, channel distortion, and changes in speech. Evaluating each condition separately does not

Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RL

arXiv:2609.38065v1 Announce Type: cross Abstract: Training agents to follow arbitrary instructions is an important goal of multi-task reinforcement learning (RL). Linear temporal logic (LTL) provides

Learning as Deepfakes Evolve: RF-Prompt for Continual Audio Deepfake Detection

arXiv:2609.37586v1 Announce Type: cross Abstract: Continual audio deepfake detection requires learning newly emerging deepfake methods while retaining discrimination of previously encountered speech.

Learning Nonlinear Finite Element Solution Operators using Multilayer Perceptrons and Energy Minimization

arXiv:2412.04596v3 Announce Type: replace Abstract: We develop and evaluate a method for learning solution operators to nonlinear problems governed by partial differential equations (PDEs). The approa

researcharxiv-cs-lg

LLM unbranding: Erasing Commercial Identity while Preserving Generic Utility

model-releasesarxiv-cs-cl

arXiv:2609.37127v1 Announce Type: new Abstract: Establishing unbranding as a critical practice to prevent visual logos from acquiring negative connotations is standard in image generation. Large Langu

Lost in Conversation or Lost in Translation? Diagnosing Multi-Turn Degradation in RAG

arXiv:2609.36700v1 Announce Type: cross Abstract: When conversing with large language models (LLMs), users often begin with a simple question and build towards a multi-hop question through follow-up t

Multichannel Audio Quality Assessment: Extending Pretrained Perceptual Models to Spatial Audio

arXiv:2609.37116v1 Announce Type: cross Abstract: Accurate perceptual quality assessment is essential for evaluating and optimizing spatial audio, where perceived quality depends on both signal fideli

NIDAR: NIR-Guided Intrinsic Decomposition for Scalable Scene-Agnostic LiDAR Intensity Reconstruction

arXiv:2609.36878v1 Announce Type: new Abstract: LiDAR return intensity provides complementary surface-response cues for robotic perception and state estimation, yet many simulation pipelines omit it o

researcharxiv-cs-ro

Retrieval Sensitivity to Identity Signals in Queries

arXiv:2609.36534v1 Announce Type: new Abstract: Dense retrievers decide which documents reach users and the language models that use them, yet they are typically evaluated with neutral queries. We ask

safetyarxiv-cs-cl

RoboChrono: A Real Robot Benchmark for Streaming Task Understanding

model-releasesarxiv-cs-ro

arXiv:2609.36605v1 Announce Type: new Abstract: Understanding ongoing robot manipulation requires models to interpret visual observations in relation to interaction history and task progress. We intro

SQUARE: Structured Quantum Representation Adapters as Compact Quadratic Feature Maps for Frozen Language Models

arXiv:2609.37134v1 Announce Type: cross Abstract: Frozen language models (LMs) are increasingly used as fixed feature extractors for downstream reranking, scoring, and preference modeling, raising a p

Think Before You Score: Thinking Reward Model for Visual Generation

arXiv:2609.37372v1 Announce Type: new Abstract: Visual reward models are essential for evaluating and improving visual generation models, yet existing approaches typically map task conditions and cand

safetyarxiv-cs-cv