Search: “evaluation”

TraceGraph: Shared Decision Landscapes for Diagnosing and Improving Agent Trajectories

arXiv:2605.31308v1 Announce Type: new Abstract: Agent benchmarks increasingly record rich interaction trajectories, yet evaluation often reduces each rollout to a pass rate or reward score. We introdu

1 Jun 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Unmute the Patch Tokens: Rethinking Probing in Multi-Label Audio Classification

DGX agent

arXiv:2509.24901v4 Announce Type: replace-cross Abstract: Although probing frozen models has become a standard evaluation paradigm, self-supervised learning in audio defaults to fine-tuning when pursu

model-releasesarxiv-cs-lg

1 Jun 2026

A Composable Multimodal Framework for cine CMR-Text-Driven Prediction of Heart Failure Outcomes

arXiv:2502.16548v3 Announce Type: replace-cross Abstract: Objective. Heart failure is one of the leading causes of death worldwide, with millions of deaths each year, according to data from the World

Beyond Attack Success Rate: Temporal Logit Observability for LLM Safety Failures

arXiv:2605.29629v1 Announce Type: new Abstract: Attack Success Rate (ASR) evaluates each jailbreak with a single yes/no label at the end of generation, telling us whether a failure happened but not ho

BioRefusalAudit: Auditing Biosecurity Refusal Depth Using General and Domain-Fine-Tuned Sparse Autoencoders

arXiv:2605.30162v1 Announce Type: new Abstract: Biosecurity evaluations of language models typically ask whether models produce hazardous output. This paper asks a complementary question: when a model

CLUBench: A Clustering Benchmark

model-releasesarxiv-cs-lg

arXiv:2605.29933v1 Announce Type: new Abstract: Clustering is a fundamental problem in data science with a long-standing research history, yielding numerous insightful algorithms. Despite this progres

Do Physics Foundation Models Learn Generalizable Physics? A Bias-Aware Benchmark Across Physical Regimes and Distribution Shifts

arXiv:2605.29283v1 Announce Type: cross Abstract: Recent physics foundation models claim general spatiotemporal forecasting ability, yet their evaluations often collapse performance into a single aver

Local Ai

EXACT-MPPI: Exact Signed-Distance Navigation for Arbitrary-Footprint Robots from Point Clouds via Path Integral Control

arXiv:2605.29663v1 Announce Type: new Abstract: Ground robots often carry payloads, implements, or other attachments that turn their effective footprint into complex, non-convex shapes. Navigating saf

local-aiarxiv-cs-ro

Gram: Assessing sabotage propensities via automated alignment auditing

arXiv:2605.30322v1 Announce Type: cross Abstract: We introduce Gram, an automated alignment auditing framework to assess the propensity of AI agents to engage in sabotage. We evaluate Gemini models ac

Latent Performance Profiling of Large Language Models

arXiv:2605.30018v1 Announce Type: new Abstract: Large language models (LLMs) frequently achieve impressive scores on standardized benchmarks, yet accuracy alone offers a limited view of their capabili

LLMBridge: An LLM Pipeline for End-to-end Referential Bridging Resolution in English

arXiv:2605.29048v1 Announce Type: new Abstract: In this paper, we introduce LLMBridge, a new LLM based system for the task of end-to-end referential bridging resolution in English. Our bridging resolu

researcharxiv-cs-cl

LoCoT2V-Bench: Benchmarking Long-Form and Complex Text-to-Video Generation

arXiv:2510.26412v3 Announce Type: replace-cross Abstract: Recent advances in text-to-video generation have achieved impressive performance on short clips, yet evaluating long-form generation under com

ReasonBreak: Probing Vulnerabilities in Reasoning-Enabled Vision-Language-Action Models for Autonomous Driving

model-releasesarxiv-cs-lg

arXiv:2605.29114v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models with integrated reasoning have been proposed for end-to-end autonomous driving, assuming a tight coupling between

Redundant or Necessary? A Benchmark for Detecting Redundant Steps in Agent Trajectories

arXiv:2605.29893v1 Announce Type: new Abstract: LLM-based agents have demonstrated strong capabilities in solving complex tasks through multi-step reasoning and tool use. However, existing evaluation

Selective QA over Conflicting Multi-Source Personal Memory: A Diagnostic Testbed and Method Comparison

arXiv:2605.30087v1 Announce Type: new Abstract: Emerging personal AI agents are moving toward persistent, multi-source memory. This creates an evaluation problem: systems must decide how to use confli

TIMEGATE: Sustainable Time-Boxed Promotion Gates for Continual ML Adaptation Under Resource Constraints

arXiv:2605.29183v1 Announce Type: cross Abstract: As machine learning(ML) systems evolve to continual adaptation, each re-training cycle uses compute, annotation, and energy. We introduce TIMEGATE, a

Toward Ethical Facial Age Estimation: A Generalized Zero-Shot Benchmark Without Training on Children's Data

arXiv:2605.29230v1 Announce Type: cross Abstract: Age estimation from facial images typically relies on training data that includes images of minors, a practice that raises serious ethical, legal, and

Applications

When Does Persona Prompting Actually Help? A Retrieval and Metric Analysis of Expert Role Injection in LLMs

arXiv:2605.29420v1 Announce Type: new Abstract: Persona prompting is widely used to steer large language models, yet its practical value remains unclear. Prior work often evaluates persona prompting u

applicationsarxiv-cs-ai

Applications

Analyzing Cancer Patients' Experiences with Embedding-based Topic Modeling and LLMs

arXiv:2601.12154v2 Announce Type: replace Abstract: This study investigates the use of neural topic modeling and LLMs to uncover meaningful themes from patient storytelling data, to offer insights tha

applicationsarxiv-cs-cl

Asking Is Not Enough: Protocol Sensitivity in LLM Confidence Calibration

arXiv:2605.27752v1 Announce Type: new Abstract: LLM confidence calibration is often evaluated by comparing two signals: token-probability scores and verbalized confidence. These signals are sometimes

Auditable Decision Models with Learned Abstention and Real-Time Steering

arXiv:2605.27768v1 Announce Type: new Abstract: Production AI systems often operate with incomplete, conflicting, or insufficient evidence. Forced classifiers collapse such cases into action labels, w

Benchmarking Fairness in Spiking Neural Networks: Data Bias, Spurious Features, and Hardware Effects

arXiv:2605.27407v1 Announce Type: cross Abstract: Evaluating fairness in Spiking Neural Networks (SNNs) demands rigorous benchmarks that reflect real-world complexities, yet existing assessments remai

BenGER: Benchmarking LLM Systems on Subsumption-Based Legal Reasoning in German Law

arXiv:2605.28183v1 Announce Type: cross Abstract: We introduce the BenGER (Benchmark for German Law) dataset for evaluating LLM systems on subsumption-based legal reasoning in German law. The BenGER d

Better Accuracies, Worse Reasoning: A Step-Level Audit of Medical Chain-of-Thought Distillation

arXiv:2605.28301v1 Announce Type: new Abstract: Chain-of-thought (CoT) distillation trains a smaller model to imitate a teacher's reasoning trace, but it is typically evaluated by final-answer metrics

Breaking the Script Barrier: Enabling Automatic Alignment for PoS-based ASR Error Analysis in Non-Latin Scripts

arXiv:2605.28438v1 Announce Type: new Abstract: Automatic Speech Recognition (ASR) systems are commonly evaluated using aggregate metrics such as Word Error Rate (WER), which do not capture the lingui

safetyarxiv-cs-cl

Build a test suite that grows with your agent with dataset management in Amazon Bedrock AgentCore

model-releasesaws-ml-blog

Agent evaluation is most powerful when you combine fast-moving online signals with stable offline baselines. To understand whether your agent is truly improving over time, you need a fixed benchmark a

Do Clinical Models Change Treatment Decisions?

arXiv:2605.28129v1 Announce Type: new Abstract: Clinical foundation models are evaluated with factual or exam-style medical QA, but treatment decisions must change when patient context changes. We int

EgoBench: An Interactive Egocentric Multimodal Benchmark for Tool-Using Agents

arXiv:2605.27820v1 Announce Type: new Abstract: As AI agents increasingly operate in open, real-world environments, they require a deep synergy of multimodal perception, tool invocation with multi-hop

Evolving and Detecting Multi-Turn Deception using Geometric Signatures

arXiv:2605.27671v1 Announce Type: cross Abstract: Safety defenses for large language models (LLMs) are typically trained and evaluated on single-turn prompts, yet real attacks often unfold as indirect

safetyarxiv-cs-lg

LLM Zeroth-Order Fine-Tuning is an Inference Workload

model-releasesarxiv-cs-lg

arXiv:2605.28760v1 Announce Type: new Abstract: Zeroth-order (ZO) fine-tuning is attractive for large language models because it replaces backpropagation with forward objective evaluations. Existing i

MUSE: Benchmarking Manufacturable, Functional, and Assemblable Text-to-CAD Generation

arXiv:2605.28579v1 Announce Type: new Abstract: Large language models (LLMs) have recently advanced text-driven 3D generation, yet Text-to-CAD remains far from supporting industrial product design. Ex

PubMedCausal: A Span-Level Annotated Corpus for Causal Relation Extraction in Biomedical Text

arXiv:2605.28363v1 Announce Type: new Abstract: Causal relation extraction (CRE) is central to biomedical text mining, but current resources often conflate causal relations with broader associations,

SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoning

arXiv:2605.28074v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) mitigates LLM hallucinations but introduces a critical vulnerability: corpus integrity. We present SilentRetrieva

researcharxiv-cs-cl

StoryMI: Steerable Multi-Agent Therapeutic Dialogue Generation

arXiv:2605.27393v1 Announce Type: cross Abstract: Large language models (LLMs) can generate fluent dialogue, but prior works lack situational grounding, dynamic strategy control, and evaluation aligne

Applications

Surrogate modeling for Bayesian optimization beyond a single Gaussian process

arXiv:2205.14090v2 Announce Type: replace-cross Abstract: Bayesian optimization (BO) has well-documented merits for optimizing black-box functions with an expensive evaluation cost. Such functions eme

applicationsarxiv-cs-lg

The Cases LJP Never Sees: Prosecution Decision Prediction for More Complete Criminal Liability Assessment

arXiv:2605.28464v1 Announce Type: cross Abstract: Legal Judgment Prediction (LJP) has become a core benchmark for evaluating AI in the criminal legal domain, but it only sees criminal cases that have

The Shape of Reasoning: Topological Analysis of Reasoning Traces in Large Language Models

arXiv:2510.20665v3 Announce Type: replace Abstract: Evaluating the quality of reasoning traces from large language models remains understudied, labor-intensive, and unreliable: current practice relies

When Discourse Pressures Conflict: Information Structure in Vision-Language Model Outputs

arXiv:2605.28346v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly evaluated for whether they identify the right visual content, but little is known about whether they expr

researcharxiv-cs-cl

Local Ai

Where Does Toxicity Live? Mechanistic Localization and Targeted Suppression in Language Models

arXiv:2605.27997v1 Announce Type: cross Abstract: Large language models frequently generate toxic, hateful, or harmful content, yet existing mitigation methods rely on costly retraining or output-leve

local-aiarxiv-cs-ai

ADRD-Bench: A Preliminary LLM Benchmark for Alzheimer's Disease and Related Dementias

arXiv:2602.11460v2 Announce Type: replace Abstract: Large language models (LLMs) have shown great potential for healthcare applications. However, existing evaluation benchmarks provide minimal coverag

Anchor: Mitigating Artifact Drift in Agent Benchmark Generation

arXiv:2605.26321v1 Announce Type: new Abstract: AI agents are beginning to complete valuable, long-horizon business operations tasks, but training and evaluation environments for enterprise work still

Are Video Models Zero-Shot Learners and Reasoners in Education? EduVideoBench, A Knowledge-Skills-Attitude Benchmark for Educational Video Generation

arXiv:2605.26918v1 Announce Type: new Abstract: Video generation models (VGMs) are rapidly entering classrooms, yet existing benchmarks evaluate only perceptual quality, intrinsic faithfulness, generi

Athena: Enhancing Multimodal Reasoning with Data-efficient Process Reward Models

arXiv:2506.09532v5 Announce Type: replace-cross Abstract: We present Athena-PRM, a multimodal process reward model (PRM) designed to evaluate the reward score for each step in solving complex reasonin

BESPOKE: Benchmark for Search-Augmented Large Language Model Personalization via Diagnostic Feedback

arXiv:2509.21106v2 Announce Type: replace Abstract: Search-augmented large language models (LLMs) have advanced information-seeking tasks by integrating retrieval into generation, reducing users' cogn

Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMs

arXiv:2605.27157v1 Announce Type: new Abstract: Retrieval-augmented LLMs are deployed for tasks where evidence quality determines action safety, yet evaluation protocols assume that single-turn robust

From Norms to Indicators (N2I-RAG): An Agentic Retrieval-Augmented Generation Framework for Legal Indicator Computation

arXiv:2605.26926v1 Announce Type: new Abstract: Computing legal indicators from normative texts is a key task in legal monitoring and policy evaluation, but presents significant challenges due to the

LiveK12Bench: Have Large Multimodal Models Truly Conquered High School-level Examinations?

arXiv:2605.26781v1 Announce Type: new Abstract: Advanced Large Multimodal Models (LMMs) have demonstrated impressive performance in K-12 reasoning tasks, exhibiting great promise as intelligent tutors

MedGuideX: Internalizing Decision Logic from Executable Guidelines into Large Language Models for Clinical Reasoning

arXiv:2605.26567v1 Announce Type: new Abstract: Clinical practice guidelines (CPGs) encode evidence-based decision logic that clinicians apply by evaluating patient variables, conditional criteria, an