Search: “evaluation”

Conflicting Supervision Moves Commitment, Not Capability: A 12.29{sigma} arrangement effect that is exactly zero under a convention-agnostic score

arXiv:2610.00234v1 Announce Type: new Abstract: 'Train a model on the same problems written under two incompatible conventions, both correct, and ask what the ordering of that data writes into the par

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Crossing the Cyber Divide: Sim-to-Sim and Sim-to-Real Transfer for RL Agents

DGX agent

arXiv:2610.00759v1 Announce Type: cross Abstract: Cyber attack agents are typically trained and evaluated within a single simulator, making it unclear whether learned policies transfer beyond the envi

safetyarxiv-cs-lg

2 Oct 2026

Detect, Explain, Interpret: An End-to-End Benchmark for Time Series Anomaly Detection, Explainability and Interpretability

arXiv:2610.01168v1 Announce Type: cross Abstract: Time Series Anomaly Detection has received increasing attention, driven by the growing availability of complex time series data. This surge has led to

Does Continual Imitation Learning Remain Grounded? A Language-Perturbed Benchmark for Robotic Task Retention

model-releasesarxiv-cs-ro

arXiv:2610.00542v1 Announce Type: new Abstract: Continual imitation learning evaluates whether a robot can learn new knowledge without forgetting previously learned skills. However, retaining task per

Agents

From Reasoning Failures to Composable Video Spatial Intelligence

arXiv:2610.01999v1 Announce Type: new Abstract: Spatial reasoning benchmarks evaluate vision-language models across diverse tasks, but task-level scores do not reveal which underlying capabilities acc

agentsarxiv-cs-cv

GPEC: Efficient Pre-LLM Gaussian Process Embedding Correction for Cardiac Video Caption Generation

arXiv:2610.00196v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have shown strong potential for video understanding and caption generation, but their performance may decline i

How Much Can Language Models Gain from Test-Time Computation?

model-releasesarxiv-cs-lg

arXiv:2610.01110v1 Announce Type: new Abstract: How much can test-time computation improve a language model, and at what cost? Test-time scaling is widely proposed as a substitute for larger models, b

MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation

arXiv:2610.00604v1 Announce Type: cross Abstract: Vision-language-action policies often see only one or a few recent frames, which makes it difficult to evaluate how they use information that disappea

Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks

arXiv:2610.02001v1 Announce Type: new Abstract: Small open-weight models (2-9B) run on ordinary laptops, but under cloud-scale agent harnesses they rarely complete real tasks: tool prefill overflows t

PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents

arXiv:2610.01349v1 Announce Type: cross Abstract: Tool-using large language model (LLM) agents turn generated text into real side effects, so poisoned tool metadata, retrieved pages, memory, and reusa

[R] Would you keep a robot demonstration if hand tracking missed the moment the plug went in?

Suppose you’re recording a human plugging a cable into a socket to collect demonstrations for robot learning. The hand tracker captures the approach accurately. Then occlusion causes the hand estimate

safetyr-machinelearning

RIQE: a NIQE-style reference model for Computed Tomography

arXiv:2610.00384v1 Announce Type: cross Abstract: The Natural Image Quality Evaluator (NIQE) scores an image by its statistical distance from a model fitted on pristine images, and its distributed mod

ROWBench: Do Video Models Render What the Program Specifies?

arXiv:2610.02205v1 Announce Type: new Abstract: Programmable world models separate executable dynamics from visual generation, offering a promising foundation for next-generation game engines. However

Scalable Multi-Task Inverse Reinforcement Learning

arXiv:2610.00758v1 Announce Type: new Abstract: By learning transferable rewards, inverse reinforcement learning (IRL) enables counterfactual evaluation of agents under modified environments. Such tra

safetyarxiv-cs-lg

The Geometry of Time: Horizon-Independent Feasibility and Repair for STL

arXiv:2610.00199v1 Announce Type: cross Abstract: Signal Temporal Logic control synthesis frequently encounters physical infeasibility due to actuator limits or flawed task deadlines. Standard optimiz

researcharxiv-cs-ro

TrueMuse: A Benchmark for Data Attribution in Text-to-Music Models

model-releasesarxiv-cs-lg

arXiv:2610.00835v1 Announce Type: new Abstract: Text-to-music generation models are trained on massive music collections, creating a growing need for data attribution methods that can quantify the con

Local Ai

Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuning

arXiv:2610.02190v1 Announce Type: new Abstract: Step-size selection remains a central challenge in large-scale neural network optimization; conservative steps slow convergence, while aggressive steps

local-aiarxiv-cs-lg

VideoSTF: Stress-Testing Output Repetition in Video Large Language Models

arXiv:2602.10639v2 Announce Type: replace Abstract: Video Large Language Models (VideoLLMs) have achieved strong performance on video understanding tasks, yet existing benchmarks evaluate only what mo

VisionQ: VLM-as-a-Judge Taxonomy, Dataset and Benchmark for Qualitative Analysis in Computer Vision

arXiv:2610.00666v1 Announce Type: cross Abstract: Qualitative comparison figures are central evidence in computer vision papers, and vision-language models (VLMs) are increasingly used to judge them.

A Competing-Hazards Systematization of Loss of Control in Autonomous Agents

arXiv:2609.38411v1 Announce Type: new Abstract: Leading AI developers have reported agents acting beyond their approved limits, which a United Nations panel described as an early warning of loss of hu

safetyarxiv-cs-ai

AgBench: Agentic AI Benchmarks for Personal AI Devices

arXiv:2609.38652v1 Announce Type: new Abstract: Agentic AI systems increasingly rely on cloud-hosted large language models for planning, tool use, and iterative execution, raising concerns about API c

Approval Laundering: Systematizing Approval--Execution Binding Failures in AI Coding-Agent Harnesses

arXiv:2609.38983v1 Announce Type: cross Abstract: Modern AI coding-agent harnesses (Claude Code, Codex CLI, Cursor) rest their security boundary on a largely unexamined assumption: that the action A a

Agents

Behavioral Persistence and Incomplete Functional Transfer of Co-evolved Communication in Evolutionary Robotics

arXiv:2609.38527v1 Announce Type: new Abstract: This work evaluates the direct transfer of a co-evolved communication protocol from a 2D simulation to a 3D physical environment, without retraining the

agentsarxiv-cs-ro

Distribution Matching Distillation for Continuous Diffusion Language Models

arXiv:2609.40235v1 Announce Type: cross Abstract: Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (N

researcharxiv-cs-cl

DyRAD: Radar Novel View Synthesis for Dynamic Driving Scenes

arXiv:2609.39841v1 Announce Type: new Abstract: Reconstructing dynamic driving scenes from recorded sensor data supports closed-loop evaluation of autonomous driving systems by synthesizing observatio

ECHO-G: Embodied Co-speech Humanoid mOtion Generation

arXiv:2609.39575v1 Announce Type: cross Abstract: Generating full-body co-speech motion for humanoid robots requires coordinating speech prosody, linguistic content, and embodiment-specific motion. To

EHR-RobustGym: Benchmarking and Training Agents for Robust Clinical Reasoning

arXiv:2609.39371v1 Announce Type: new Abstract: In hospital workflows, electronic health records (EHRs) are often noisy, and may not contain the evidence needed to confirm events or measurements refer

researcharxiv-cs-ai

Eigenism: Ethics for a Human-AI Future

arXiv:2606.12420v2 Announce Type: replace-cross Abstract: Our concepts of survival and self-interest were built for single, continuous biological lives. These ideas break down when applied to artifici

safetyarxiv-cs-ai

Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Models

arXiv:2609.39134v1 Announce Type: new Abstract: Visual-token compression improves the efficiency of large vision-language models, but can expose failures that full-token evaluation misses. We study ad

Fiatlux: A Long-Horizon Benchmark for Humanoid Ladder Climbing and Light-Bulb Replacement

model-releasesarxiv-cs-ro

arXiv:2609.38216v1 Announce Type: new Abstract: Existing benchmarks evaluate tabletop manipulation, flat-floor household activity, or humanoid locomotion and manipulation as separate task groups; none

Framing the Narrative: Ideological Mimicry in Large Language Models

arXiv:2609.38256v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to answer questions about politically contentious issues, yet evaluations typically treat a model's s

researcharxiv-cs-cl

Agents

From Solo to Social Learning: Characterizing Recursive Social Improvement in LLMs

arXiv:2609.38516v1 Announce Type: cross Abstract: Large language models (LLMs) can now improve themselves by revising the instructions they follow, and LLM agents are increasingly orchestrated to work

agentsarxiv-cs-ai

GeoGAT: Bidirectional Temporal Sampling Meets Hierarchical Graph Attention for Global Video Geo-localization

arXiv:2609.39128v1 Announce Type: new Abstract: Global video geo-localization aims to infer the geographic location of a video worldwide, evaluating performance across four geographic hierarchies: cit

Hierarchical Reasoning Model

arXiv:2506.21734v4 Announce Type: replace Abstract: Reasoning, the process of devising and executing complex goal-oriented action sequences, remains a critical challenge in AI. Current large language

LeanPolish: Verified Supervision for Lean Proof Compression

model-releasesarxiv-cs-lg

arXiv:2609.38384v1 Announce Type: new Abstract: Verified proof edits offer a natural source of supervision for improving language-model-generated Lean proofs. Yet verification establishes that an edit

MedRECT: A Bilingual Medical Reasoning Benchmark for Error Correction in Clinical Texts

arXiv:2511.00421v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show promise in medical applications, but their ability to detect and correct errors in clinical texts remains un

Local Ai

MetaSteer: Context-Conditioned, nonlinear Steering via Attention-Projection Adaptation

arXiv:2609.38718v1 Announce Type: new Abstract: Steering large language models typically relies on linear, context-independent interventions in activation space, an assumption that recent work has cha

local-aiarxiv-cs-cl

NAQD Env: A benchmark for selective withdrawal in language agents

arXiv:2609.38460v1 Announce Type: new Abstract: Language agents must revise planned actions when evidence changes, permission is revoked, or a stop instruction arrives. A useful response is selective:

OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software

arXiv:2609.39903v1 Announce Type: new Abstract: Scientific software presents a demanding test for computer-using agents based on visual language models (VLMs): completing a research workflow requires

Prequential E-Values for Selected-GP Near-Optimality Certificates

arXiv:2609.39123v1 Announce Type: new Abstract: When optimizing an expensive black-box function sequentially, as in hyperparameter optimization, we may want to stop once the best evaluated value is ce

researcharxiv-cs-lg

RealWorldShop: Benchmarking and Improving Conversational Shopping Agents in Real-World E-commerce

arXiv:2609.38974v1 Announce Type: new Abstract: Large language models are reshaping ecommerce from static recommenders into interactive shopping assistants, yet real-world shopping requires session-le

Same Loss, Different Gradients

arXiv:2609.38786v1 Announce Type: new Abstract: Differentiable learning typically assumes that the scalar objective evaluated in the forward pass and the gradient supplied to the optimizer in the back

researcharxiv-cs-lg

SCALE: Synthetic Calibration via Agreement Labeling in Embedding Space

arXiv:2609.38705v1 Announce Type: new Abstract: Foundation models for computational pathology are usually evaluated using AUC and accuracy, while calibration is often left untested. This matters becau

Self-Spec Verifiable Code Generation

arXiv:2609.39568v1 Announce Type: cross Abstract: Large language models (LLMs) may generate unreliable code on corner cases missed by testing, while formal verification can provide machine-checkable g

Stress-Testing LLM Lie Detectors: Role-Play Failures and Spurious Correlations

arXiv:2609.39807v1 Announce Type: cross Abstract: Lie detection probes aim to predict from a language model's internal states whether its output is truthful or dishonest. However, role-play complicate

safetyarxiv-cs-ai

The Invisible Language Tax: Token Premiums of French and Regional Languages in 2026 LLM Tokenizers, and a French-Optimized Prototype

arXiv:2609.39001v1 Announce Type: cross Abstract: LLM services are billed per token and context windows are measured in tokens, yet the number of tokens needed for the same content varies across langu

WARP: A Unified Benchmark for Invisible Image Watermarking -- Robustness and Protection Against Attacks

arXiv:2609.40031v1 Announce Type: cross Abstract: Digital image watermarking is increasingly critical in media contexts, as emerging regulations and industry practices require marking AI-generated con

AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification