Search: “evaluation”

OncoTraj: a public benchmark for longitudinal resistance prediction in EGFR-mutant non-small-cell lung cancer on osimertinib

arXiv:2606.11144v1 Announce Type: new Abstract: Resistance to first-line osimertinib in EGFR-mutant non-small-cell lung cancer (NSCLC) is the canonical example of predictable clonal evolution under th

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

POPSICLE: Benchmark Datasets for Segmentation and Localization in CryoET

DGX agent

arXiv:2606.10255v1 Announce Type: cross Abstract: Cryo-electron tomography (cryoET) has emerged as a powerful tool in structural and cellular biology by enabling direct visualization of macromolecular

model-releasesarxiv-cs-cv

10 Jun 2026

SpeechJBB: Probing Safety Alignment and Comprehension in Large Audio Language Models under Code-Switched Speech

arXiv:2606.06037v2 Announce Type: cross Abstract: Large audio language models (LALMs) are increasingly deployed in real-world applications, yet their safety alignment is still primarily evaluated on m

safetyarxiv-cs-cl

TaCarla: A comprehensive benchmarking dataset for end-to-end autonomous driving

arXiv:2602.23499v4 Announce Type: replace-cross Abstract: Collecting a high-quality dataset is a critical task that demands meticulous attention to detail, as overlooking certain aspects can render th

V-REX: Benchmarking Exploratory Visual Reasoning via Chain-of-Questions

arXiv:2512.11995v2 Announce Type: replace-cross Abstract: While many vision-language models (VLMs) are developed to answer well-defined, straightforward questions with highly specified targets, as in

VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Models

arXiv:2606.10568v1 Announce Type: new Abstract: Vision-language-action (VLA) models have shown strong promise for robotic manipulation, but their reliability at test time remains limited by one-shot a

applicationsarxiv-cs-ro

What Demonstration Curation Metrics Do to Your Policy

arXiv:2606.10229v1 Announce Type: cross Abstract: We study whether demonstration-curation metrics that detect defective training episodes also improve the downstream behavior-cloning policy that train

AI Scientists Are Only as Good as Their Evidence: A Stratified Ablation of Proprietary Data and Reasoning Skills in Drug-Asset Valuation

arXiv:2606.09556v1 Announce Type: new Abstract: AI Scientist agents are often evaluated as if capability were mainly a function of model quality, prompting, or reasoning scaffolds. We test a different

ATM: Action-Consistency Transfer Matrix for Diagnosing and Improving Latent World Models

arXiv:2606.09028v1 Announce Type: cross Abstract: Latent world models are increasingly used for control and goal-conditioned planning, yet assessing whether their learned representations are useful fo

AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs

arXiv:2606.07643v1 Announce Type: cross Abstract: Recent advances in Omni-Multimodal Large Language Models (Omni-MLLMs) have enabled strong integration of vision, audio, and language. However, their a

Beyond Accuracy: Interpreting Topic Representation in Suicide Ideation Detection Models

arXiv:2606.07714v1 Announce Type: cross Abstract: Suicide ideation detection models are typically evaluated using aggregate performance metrics, yet little is known about how they internally represent

safetyarxiv-cs-ai

Beyond Agent Architecture: Execution Assumptions and Reproducibility in LLM-Based Trading Systems

arXiv:2606.08285v1 Announce Type: new Abstract: Large language models (LLMs) and agentic systems are increasingly proposed for financial trading, yet their reported performance remains difficult to co

Beyond Pass/Fail: Using Process Mining to Understand How LLMs Resist (and Fail) Red Team Attacks

arXiv:2606.07833v1 Announce Type: cross Abstract: Standard AI red teaming evaluations reduce adversarial campaigns to a single binary outcome, attack success rate (ASR), not taking into account the se

CHIMERA-Bench: A Benchmark Dataset for Epitope-Specific Antibody Design

arXiv:2603.13431v3 Announce Type: replace-cross Abstract: Computational antibody design has seen rapid methodological progress, with dozens of deep generative methods proposed in the past three years,

CTS-Bench: Benchmarking Graph Coarsening Trade-offs for GNNs in Clock Tree Synthesis

arXiv:2602.19330v2 Announce Type: replace Abstract: Graph Neural Networks (GNNs) are increasingly explored for physical design analysis in Electronic Design Automation, particularly for modeling Clock

Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control

arXiv:2512.01467v2 Announce Type: replace Abstract: Controlling autonomous systems under real-world conditions often requires policies that can be evaluated with low latency and minimal energy consump

agentsarxiv-cs-lg

DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Driving

model-releasesarxiv-cs-cv

arXiv:2606.08525v1 Announce Type: new Abstract: Reward models play a pivotal role in reinforcement learning (RL) and multi-modal trajectory selection for autonomous driving. However, acquiring such re

Echo-Memory: A Controlled Study of Memory in Action World Models

arXiv:2606.09803v1 Announce Type: new Abstract: We present extbf{Echo-Memory}, a controlled study of memory mechanisms in action-conditioned world models. These models generate multi-segment videos fr

researcharxiv-cs-cv

ERBench: A Benchmark and Testsuite for Equation Discovery Algorithms

arXiv:2606.09276v1 Announce Type: new Abstract: Equation discovery aims to automate the discovery of scientific models in the form of mathematical equations from data. Technically, equation discovery

Executable World Models for ARC-AGI-3 in the Era of Coding Agents

arXiv:2605.05138v2 Announce Type: replace Abstract: We evaluate an initial coding-agent system for ARC-AGI-3 in which the agent maintains an executable Python world model, verifies it against previous

Fast LLM-Based Semantic Filtering: From a Unified Framework to an Adaptive Two-Phase Method

arXiv:2606.08090v1 Announce Type: cross Abstract: Evaluating a natural-language yes/no predicate over a document corpus under an accuracy target - the semantic filter - is a cornerstone of LLM-based d

safetyarxiv-cs-ai

From 0-to-1 to 1-to-N: Reproducible Engineering Evidence for MetaAI Recursive Self-Design

arXiv:2606.09663v1 Announce Type: new Abstract: Recursive self-design refers to AI-assisted modification of the mechanisms by which an AI system is built, evaluated, and improved. This paper treats Me

How Much Capacity Does EEG Denoising Need? Ultra-Compact Networks reveal Benchmark Saturation and Metric-Utility Gap

arXiv:2606.08594v1 Announce Type: new Abstract: Deep learning EEG denoising architectures have scaled from tens of thousands to tens of millions of parameters, yet no prior study has isolated model ca

IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation

model-releasesarxiv-cs-cv

arXiv:2601.04498v2 Announce Type: replace-cross Abstract: Infographics are composite visual artifacts that combine data visualizations with textual and illustrative elements to communicate information

Larch: Learned Query Optimization for Semantic Predicates

arXiv:2606.07923v1 Announce Type: cross Abstract: With the advent of Large Language Models (LLMs), many database systems introduced semantic operators that enabled analytical queries over unstructured

LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty

arXiv:2503.18314v5 Announce Type: replace-cross Abstract: We present LoTUS, a novel Machine Unlearning (MU) method that eliminates the influence of training samples from pre-trained models, avoiding r

MedicalRec: Medical recommender system for image classification without retraining

arXiv:2606.07553v1 Announce Type: cross Abstract: The emergence of machine learning and deep learning has revolutionized the efficiency of diagnostic, therapeutic, and administrative systems in health

No Free Lunch for Synthetic Images under Data Scarcity Conditions

arXiv:2606.07640v1 Announce Type: cross Abstract: This study investigates the trade-offs between fidelity, privacy, and utility in synthetic data generation under conditions of data scarcity and priva

PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents

arXiv:2606.08106v1 Announce Type: new Abstract: Self-evolving agents improve by repeatedly proposing changes to their own prompts, skills, or workflows and keeping those that score higher on a small h

Partial Identification under Missing Data Using Weak Shadow Variables from Pretrained Models

arXiv:2602.16061v2 Announce Type: replace-cross Abstract: Estimating population quantities such as mean outcomes from user feedback is fundamental to platform evaluation and social science, yet feedba

researcharxiv-cs-lg

Principled Agent Debate: Adversarial Arbitration for Sycophancy Reduction in Large Language Models

arXiv:2606.07532v1 Announce Type: cross Abstract: RLHF-trained models are systematically biased toward agreement over accuracy, a structural property of the training process. We present Principled Age

Sci-Rho: A Multilingual Visually-Grounded Symbolic Benchmark for STEM Problems

arXiv:2606.08034v1 Announce Type: cross Abstract: Symbolic benchmarks have emerged as a key approach to assess model robustness under minor modifications to STEM-related questions. However, existing s

Tutorials

Statistical Decision Theory with Counterfactual Loss

arXiv:2505.08908v3 Announce Type: replace-cross Abstract: Many researchers apply classical statistical decision theory to evaluate treatment choices and learn optimal policies. However, because this f

tutorialsarxiv-cs-lg

Hardware

Toward Compiler World Models: Learning Latent Dynamics for Efficient Tensor Program Search

arXiv:2606.09312v1 Announce Type: new Abstract: Tensor program optimization is essential for modern machine learning systems, but its search space is enormous. Existing auto-schedulers reduce measurem

hardwarearxiv-cs-lg

Training-Inference Kernel Contracts: Bounding Divergence in Post-Training and Deployment

arXiv:2606.07581v1 Announce Type: cross Abstract: A modern post-training pipeline often writes one symbol for its policy, pi_theta, while evaluating it through two different programs: a training kerne

safetyarxiv-cs-ai

WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces

arXiv:2606.09426v1 Announce Type: new Abstract: Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and ext

Who Earns the Safety? Intervention-Aware Quantum Predictive Control with Safety Attribution

arXiv:2606.09778v1 Announce Type: cross Abstract: Hard safety filters are increasingly placed downstream of learned controllers to guarantee constraint satisfaction at run time. Yet a filtered control

A Held-Out Transition-Pair Falsifier for Long-Horizon Non-Abelian State Tracking

arXiv:2606.07254v1 Announce Type: new Abstract: State tracking exposes a sharp limitation of sequence models: the relevant signal is often not a summary of observed tokens, but an ordered latent state

A Rolling-Window Framework for Churn Prediction and Behavioral Driver Identification

arXiv:2606.06776v1 Announce Type: new Abstract: Customer churn prediction is a central task in customer analytics, particularly in non-contractual, pay-per-use service environments where disengagement

applicationsarxiv-cs-lg

AI-Driven Test Case Generation from Natural Language Requirements: A Survey of Techniques and Research Gaps

arXiv:2606.06563v1 Announce Type: cross Abstract: Software testing is critical for verifying that systems meet specified requirements, yet remains among the most time-consuming and expensive activitie

Analysing Differences in Persuasive Language in LLM-Generated Text: Uncovering Stereotypical Gender Patterns

arXiv:2601.05751v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for everyday communication tasks, including drafting interpersonal messages intended to inf

DialDefer: A Framework for Detecting and Mitigating LLM Dialogic Deference

arXiv:2601.10896v2 Announce Type: replace Abstract: LLMs are increasingly used as third-party judges, yet their reliability when evaluating speakers in dialogue remains poorly understood. We show that

researcharxiv-cs-cl

MacArena: Benchmarking Computer Use Agents on an Online macOS Environment

arXiv:2606.06560v1 Announce Type: cross Abstract: Computer-use agents (CUAs) operate graphical user interfaces (GUIs) through vision and control primitives, and their capabilities have advanced rapidl

Mind the Gap: Disentangling Performance Bottlenecks in Video Instance Segmentation

arXiv:2606.07394v1 Announce Type: new Abstract: In Video Instance Segmentation (VIS), classification, segmentation, and tracking objectives are jointly evaluated, but their individual contributions to

researcharxiv-cs-cv

Mitosis Detection in the Wild: Multi-Tumor and Context-Aware Generalization in the MIDOG 2025 Challenge

arXiv:2606.07368v1 Announce Type: cross Abstract: Automated mitosis detection is a well-established task in computational pathology. While previous benchmarks focused on scanner-induced domain shift,

mmPISA-bench: Do LLMs Reason Equally Well Across 43 Languages?

model-releasesarxiv-cs-cl

arXiv:2606.07069v1 Announce Type: new Abstract: We introduce mmPISA-bench, a compact high-quality multilingual reasoning benchmark derived from the OECD Programme for International Student Assessment

RealDocBench: A Benchmark for Field-Level QA and Layout Understanding on Real-World Regulated Documents

model-releasesarxiv-cs-cv

arXiv:2606.07401v1 Announce Type: new Abstract: Document parsing systems are increasingly deployed in high-stakes, regulated workflows such as mortgage underwriting, financial reporting, supply-chain

ScenicRules: An Autonomous Driving Benchmark with Multi-Objective Specifications and Abstract Scenarios

arXiv:2602.16073v2 Announce Type: replace-cross Abstract: Developing autonomous driving systems for complex traffic environments requires balancing multiple objectives, such as avoiding collisions, ob