Search: “evaluation”

More Methods, Better Agents? A Reality Check on Self-Distillation in Agentic Reinforcement Learning

arXiv:2610.19789v1 Announce Type: new Abstract: Self-distillation in agentic reinforcement learning has produced a growing number of methods. However, existing evaluations often reuse published baseli

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Physics: Benchmarking Foundation Models on University-Level Physics Problem Solving

DGX agent

arXiv:2503.21821v2 Announce Type: replace Abstract: We introduce Physics, a benchmark for evaluating foundation models on advanced physics problem solving. It contains 1034 annotated, open-ended probl

model-releasesarxiv-cs-ai

15 Oct 2026

ProCARE: Real-World Study Automation via Profile-Grounded Evidence Contracts

arXiv:2610.16175v1 Announce Type: new Abstract: Real-world studies (RWS) analyze routinely collected healthcare data, such as electronic health records and insurance claims, to estimate treatment effe

PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience

arXiv:2606.18060v2 Announce Type: replace Abstract: As Large Language Model based agents enter autonomous scientific research, their ability to resist pseudoscience becomes increasingly important. Oth

PXRD-Bench: How Close Are We to Determining Crystal Structures from Powder Diffraction?

arXiv:2610.16480v1 Announce Type: new Abstract: Determining crystal structures from powder X-ray diffraction (PXRD) would enable routine characterization of newly synthesized materials, including thos

Local Ai

RA^2: Retain-Anchored Attraction Preserves Forget-Adjacent Utility in LLM Unlearning

arXiv:2610.19385v1 Announce Type: new Abstract: Large Language Model (LLM) unlearning is often evaluated primarily by target suppression and clean retain utility. These metrics can miss a local failur

local-aiarxiv-cs-lg

Recognizing Dialogue Ambiguity via Candidate Set Visual Grounding

arXiv:2610.19723v1 Announce Type: new Abstract: Interpreting an ambiguous utterance in real-world interactions often requires context beyond a single utterance. When the utterance refers to objects in

Right Answer, Wrong Reason: Accuracy, Consistency, and Consensus Are Misleading Indicators of LLM Faithfulness in Clinical Decision Support

arXiv:2609.32817v2 Announce Type: replace Abstract: Clinical Large Language Models (LLMs) achieve strong medical-exam accuracy; however, a correct answer does not guarantee that the explanation names

safetyarxiv-cs-ai

StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-Judge

arXiv:2610.16814v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as judges to assign scalar scores that drive downstream decision-making, including tasks such as pa

The Anatomy of a Blind Spot: What an LLM Judge Must Be Shown, and How Much

arXiv:2610.15938v1 Announce Type: new Abstract: Evaluation of text generated by large language models (LLMs) is increasingly delegated to LLMs acting as judges, yet what such a judge must be shown rem

researcharxiv-cs-ai

The Silent Failures in Agentic Data Science

arXiv:2610.14115v1 Announce Type: new Abstract: Large language models have significantly lowered the barrier to writing code. Naturally, this has also led to the development of data science agents tha

agentsarxiv-cs-ai

Towards a Benchmark for Fire Safety Code Interpretation in the Hong Kong AEC Industry

arXiv:2610.16127v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly explored in the Architecture, Engineering, and Construction (AEC) domain, yet their reliability in regul

Upstream Influence, Downstream Re-Anchoring in LLM Agent Chains

arXiv:2610.17723v1 Announce Type: new Abstract: People act on evaluative judgments produced by LLM agents, and in multi-agent systems those judgments pass through several agents before reaching a pers

agentsarxiv-cs-ai

When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO

arXiv:2610.20089v1 Announce Type: new Abstract: Pairwise preference feedback has become a widely adopted supervision signal for reinforcement learning in video generation models, as pairwise compariso

safetyarxiv-cs-cv

A CaLLM Judge is a Better Judge: Mitigating Bias in LLM-as-a-judge

arXiv:2610.18108v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as judges to predict which of two candidate answers a human evaluator would prefer, providing a scala

safetyarxiv-cs-lg

APT: Feedback-Driven Agentic Post-Training for Text-to-Image Generation

arXiv:2610.18097v1 Announce Type: new Abstract: Post-training is crucial for improving text-to-image models, yet existing pipelines typically rely on manually curated training data and fixed training

CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT

arXiv:2610.14226v1 Announce Type: new Abstract: Purpose : To develop and evaluate CAPTOR, a deep-learning vision-language foundation model for oncologic chest-abdomen-pelvis (CAP) CT interpretation. M

CESNET-TimeSeries26: A Multi-Year Network Traffic Time Series Dataset for Annual Seasonality, Concept Drift, and Long-Term Anomaly Detection

model-releasesarxiv-cs-lg

arXiv:2610.14799v1 Announce Type: cross Abstract: Forecast-based models are fundamental to network traffic management and anomaly detection. However, it is essential to evaluate these forecasting mode

CineRecon: Benchmarking Frontier Models for Agentic Cinematic Reconstruction

arXiv:2610.16577v1 Announce Type: new Abstract: Reconstructing reference videos as editable dynamic 3D scenes is an important path toward controllable video generation for filmmaking: these scenes pro

PerturbBridge: Direct Phenotypic Supervision for Cellular Perturbation Generation

arXiv:2610.17066v1 Announce Type: new Abstract: Virtual cell models should capture phenotypes induced by specific perturbations. However, image-based models are typically trained and evaluated to matc

PLDiv: Measuring Dataset Diversity and Semantic Collapse via Persistence Landscapes

arXiv:2610.14031v1 Announce Type: cross Abstract: Modern data pipelines increasingly combine human-created and human-curated data with AI-generated data for model training and evaluation. As datasets

ResearchFlow-Eval: Tracing the Execution Gap in End-to-End Multi-Claim Autoresearch

arXiv:2610.18088v1 Announce Type: cross Abstract: Autoresearch systems increasingly undertake entire research projects. In practice, scientific research requires systems to pursue multiple interdepend

agentsarxiv-cs-lg

Applications

RFWeave: Learning Wireless Propagation Across Scenes and Transceiver Configurations

arXiv:2610.16656v1 Announce Type: cross Abstract: Wireless propagation modeling is foundational to the evaluation and design of modern communication systems. To achieve accurate wireless propagation p

applicationsarxiv-cs-lg

Same Goal, Different Words: Spatial Instruction Sensitivity in World-Action Models

arXiv:2610.16242v1 Announce Type: new Abstract: World-action models should preserve the intended manipulation across equivalent descriptions of a spatial goal. We evaluate this consistency in 864 roll

researcharxiv-cs-ro

SoK: Rethinking Membership Inference in Tabular Machine Learning

arXiv:2610.18486v1 Announce Type: new Abstract: Membership Inference Attacks (MIAs) are currently the dominant approach to evaluating privacy in machine learning applications. Despite their significan

researcharxiv-cs-lg

Spatiotemporal Recognition of Abdominal Wall Layers During Optical Trocar Entry

arXiv:2610.14512v1 Announce Type: new Abstract: Objective. To evaluate whether spatial context and temporal modeling improve tissue recognition during optical trocar entry, where multiple abdominal-wa

SyncReward: Human-Aligned Synchronization Reward for Audio-Visual Generation

arXiv:2610.16338v1 Announce Type: new Abstract: Recent years have witnessed rapid progress in joint audio--visual generation. However, audio--visual synchronization remains a challenge, as generated s

Towards a Systematic Study of Embodied System 2 over Full Task Execution Cycles

model-releasesarxiv-cs-ro

arXiv:2610.17657v1 Announce Type: new Abstract: An embodied brain, or System 2, must turn an abstract goal into actions while discovering relevant scene conditions and checking whether executed skills

Video-Arena: A Scalable Agentic Video Benchmark for Spatiotemporal Capability

arXiv:2610.14158v1 Announce Type: new Abstract: Understanding and tracking a dynamic world are fundamental to visual intelligence. Existing video benchmarks provide limited insight into whether models

Build long-horizon agents with Grok 4.7 on Amazon Bedrock

Grok 4.7 is now available on Amazon Bedrock. In this post, we build a tool-calling agent on Grok 4.7 and evaluate it on three reasoning tasks at four reasoning effort levels, measuring correctness and

agentsaws-ml-blog

13 Oct 2026

DigitalOcean Inference Router Enhancements: Performance Tuned Routing, No Rules to Write

model-releasesdigitalocean

Every few weeks, a new model steps up and claims best-in-class at something. The launch chart makes the case, the link makes the rounds, and your team is back to the question nobody has time for: shou

13 Oct 2026

EU AI Act compliance for AI agents: a practical engineering guide

Learn how agent traces, human oversight, production evaluations, and log retention support EU AI Act compliance with a practical credit-assistant example. The post EU AI Act compliance for AI agents:

safetyarize-ai

13 Oct 2026

Agreement Without Alignment: Label-Free Rubric Clarification Increases LLM-Judge Consensus but Not Agreement with Humans

arXiv:2610.13303v1 Announce Type: cross Abstract: LLM judges are increasingly deployed as panels, and disagreement among panel members is often read as a symptom of an unclear rubric. We test a label-

safetyarxiv-cs-ai

Approximation of differential entropy in Bayesian optimal experimental design

arXiv:2510.00734v2 Announce Type: replace-cross Abstract: Bayesian optimal experimental design provides a principled framework for selecting experimental configurations that maximize obtained informat

researcharxiv-cs-lg

AutoBio+: Benchmarking Dexterous Manipulation with Dynamic Experimental Phenomena in Realistic Laboratories

model-releasesarxiv-cs-ro

arXiv:2610.13643v1 Announce Type: new Abstract: Automating biological laboratories requires robots to perform dexterous tool interactions under realistic visual conditions while handling dynamic exper

Localized Generative Design of Ship Hulls with Diffusion Models and High-Fidelity Assessment

arXiv:2610.12870v1 Announce Type: cross Abstract: Practical ship design requires localized refinement and performance evaluation. This paper presents a reward-directed diffusion framework for localize

Masked Diffusion Language Agents for Tool-Integrated Chemical Reasoning

arXiv:2610.13330v1 Announce Type: new Abstract: Designing tool-integrated reasoning agent systems for complex chemical tasks remains a fundamental challenge. Although recent chemical agents have demon

MC-EvoBench: Auditing Experience-Driven Self-Improvement in Minecraft Agents

arXiv:2610.13829v1 Announce Type: cross Abstract: A central challenge in embodied recursive self-improvement is enabling agents to convert interaction history into capabilities that persist beyond one

Measuring Artifact Reliance in Machine-Generated Text Detectors

arXiv:2610.13583v1 Announce Type: new Abstract: Detectors of machine-generated text score almost perfectly on scientific-literature benchmarks, implying we can reliably distinguish with these detector

researcharxiv-cs-cl

MVGraph: Benchmarking LVLMs on Evidence Composition across Multi-View Visual Graph

arXiv:2610.12961v1 Announce Type: cross Abstract: We introduce MVGraph, a benchmark for evaluating large vision-language models (LVLMs) on multi-graph reasoning where the evidence required by a query

Partially Informed Stochastic Optimization via Zeroth-Order Residual Tracking

arXiv:2610.13871v1 Announce Type: new Abstract: Many optimization problems provide direct access to only part of the gradient, while the remaining component must be inferred from function evaluations.

researcharxiv-cs-lg

perturber: A Stateless Service for Semantics-Preserving Text Perturbations

arXiv:2610.12667v1 Announce Type: new Abstract: Large language model (LLM) evaluations on clean benchmarks overstate model robustness: real inputs arrive perturbed, and these meaning-preserving varian

Pretraining and Test-Time Training Budgets Matter for ARC: A Controlled Study of VARC and TRM

arXiv:2610.13660v1 Announce Type: cross Abstract: The Abstraction and Reasoning Corpus (ARC) is increasingly used to evaluate whether neural networks can infer task-specific transformations from few d

researcharxiv-cs-ai

All Verdicts are Not Equal: Rethinking LLM Judge Reliability

arXiv:2610.12083v1 Announce Type: cross Abstract: LLM-as-a-Judge is the standard paradigm for NLP evaluation, yet its systemic reliability remains poorly understood despite being widely treated as a d

safetyarxiv-cs-ai

Beyond Imitation: A Framework and Benchmark for LLM-Assisted Peer Review

arXiv:2610.11087v1 Announce Type: new Abstract: The rapid growth of scientific publishing has strained peer review, particularly in machine learning, raising concerns about declining review quality an

From Reactive Containment to Proactive Assurance: Lessons from OpenAI, Anthropic, and Google Agent Security Incidents

arXiv:2610.12463v1 Announce Type: cross Abstract: In 2026, cybersecurity evaluations involving OpenAI, Anthropic, and Google agents reached real systems outside their authorized test scope. The paths

Tutorials

Gaze Estimation for Human-Robot Interaction: Analysis Using the NICO Platform

arXiv:2509.24001v3 Announce Type: replace Abstract: This paper evaluates the current gaze estimation methods within a human-robot interaction (HRI) context of a shared workspace scenario. We introduce

tutorialsarxiv-cs-cv

Language Models for Page-Level Layout Decisions in E-commerce Search

arXiv:2610.10920v1 Announce Type: cross Abstract: E-commerce search pages are critical touchpoints for millions of online shoppers. While traditional search engines return a ranked list of results, mo

researcharxiv-cs-cl