Search: “evaluation”

RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environments

model-releasesarxiv-cs-ro

arXiv:2509.17057v3 Announce Type: replace Abstract: We present RoboManipBaselines, an open-source software framework for imitation learning research in robotic manipulation. The framework supports the

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Safety Generalization Under Distribution Shift in Safe Reinforcement Learning: A Diabetes Testbed

DGX agent

arXiv:2601.21094v2 Announce Type: replace-cross Abstract: Safe Reinforcement Learning (RL) algorithms are typically evaluated under fixed training conditions. We investigate whether training-time safe

model-releasesarxiv-cs-ai

26 May 2026

SMDD-Bench: Can LLMs Solve Real-World Small Molecule Drug Design Tasks?

arXiv:2605.21740v2 Announce Type: replace Abstract: LLM agents have incredible potential for scientific discovery applications. However, the performance of LLM agents on real-world, small molecule dru

Teaching large language models to reason like expert diagnosticians

arXiv:2509.12194v2 Announce Type: replace Abstract: Differential diagnosis is an iterative process that integrates patient information with broader medical knowledge. Clinical case series such as the

Teaching Through Analogies: A Modular Pipeline for Educational Analogy Generation

arXiv:2605.24211v1 Announce Type: cross Abstract: Analogies help learners understand unfamiliar concepts by relating them to known concepts. Despite recent advances, large language models (LLMs) conti

TRACER: A Semantic-Aware Framework for Fine-Grained Contamination Detection in Code LLMs

arXiv:2605.24079v1 Announce Type: cross Abstract: Data contamination is a known threat to the reliability of model evaluation. However, it remains underexplored in code large language models (LLMs), w

Trait-Aware Policy Optimization for Autoregressive Multi-Trait Essay Scoring

arXiv:2605.25731v1 Announce Type: new Abstract: Multi-trait essay scoring aims to provide fine-grained evaluation of writing quality across multiple dimensions. However, how to effectively post-train

safetyarxiv-cs-cl

Tutorials

Two-Sided Time-Independent Regret for Matching Markets with Limited Interviews

arXiv:2602.12224v2 Announce Type: replace-cross Abstract: Two-sided matching platforms rely on preferences from both sides, yet participants can evaluate only a small fraction of potential partners. I

tutorialsarxiv-cs-ai

ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks

arXiv:2605.25388v1 Announce Type: new Abstract: Nucleotide sequences constitute the fundamental genetic basis of biological systems, rendering viral genomic analysis critical for biomedical advancemen

When Gradients Collide: Failure Modes of Multi-Objective Prompt Optimization for LLM Judges

arXiv:2605.26046v1 Announce Type: cross Abstract: Customizing an LLM judge to a specific task or domain often involves optimizing its prompt across multiple evaluation criteria simultaneously. Textual

researcharxiv-cs-ai

When Search Becomes Memory: Turning Robot Design Trials into Transferable Skills

arXiv:2605.25832v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as proposal generators for evolutionary robot design, yet most loops remain memoryless: simulator r

Agents

AI Assurance: A Comprehensive Testing Strategy for Enterprise AI Systems

arXiv:2605.23459v1 Announce Type: cross Abstract: Enterprise AI systems, built on large language models, retrieval pipelines and autonomous agents, introduce a class of risks that traditional software

agentsarxiv-cs-ai

Are Frontier LLMs Ready for Cybersecurity? Evidence for Vertical Foundation Models from Dual-Mode Vulnerability Benchmarks

arXiv:2605.23243v1 Announce Type: cross Abstract: We evaluate whether frontier LLMs are ready for cybersecurity through a dual-mode benchmark: white-box function-level vulnerability detection (VulnLLM

Agents

Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents

arXiv:2605.23590v1 Announce Type: new Abstract: ReAct-style agents for search-intensive, multi-step reasoning tasks rely largely on their own internal judgment to decide what evidence to seek, which r

agentsarxiv-cs-ai

DDX-TRACE: A Benchmark for Medical Diagnostic Trajectories in VLMs

arXiv:2605.23629v1 Announce Type: new Abstract: Medical diagnosis is not a single prediction from a fully specified vignette. It is a sequential workup: clinicians decide what evidence to obtain, revi

Design and Report Benchmarks for Knowledge Work

arXiv:2605.23262v1 Announce Type: new Abstract: The development of LLM agents has led to a growing body of work on knowledge-work AI, including coding, research, and healthcare. However, current knowl

Diffusion-based Denoising Beats Vanilla Score Matching in Parameter Estimation: A Theoretical Explanation

arXiv:2605.22950v1 Announce Type: cross Abstract: Score matching is an alternative to maximum likelihood estimation when the normalizing constant is unknown or too costly to evaluate. However, vanilla

Encrypted Neural Networks without Overflows

arXiv:2605.23096v1 Announce Type: cross Abstract: Fully homomorphic encryption (FHE) enables private inference by evaluating neural networks on encrypted data. In this way, we can delegate the computa

Enhancing Blood Cells Classification using Hybrid Quantum Neural Networks

arXiv:2605.23324v1 Announce Type: new Abstract: Accurate classification of microscopic blood cells is still a critical task in medical image analysis, where subtle variations and limited data can chal

researcharxiv-cs-cv

MARS: Magnitude-Aware Rank Statistics

arXiv:2605.23563v1 Announce Type: new Abstract: Comprehensive evaluation of machine learning models is the key to make sure that they perform as robustly and consistently as desired. In order to summa

Naturalistic measure of social norms alignment

arXiv:2605.23420v1 Announce Type: new Abstract: Social norms reflect shared expectations on acceptable behavior. Measuring social norms alignment remains challenging, with existing approaches typicall

safetyarxiv-cs-cl

Reading Calibrated Uncertainty from Language Model Trajectories

arXiv:2605.22864v1 Announce Type: new Abstract: The maximum softmax probability (MSP) represents a default approach when evaluating uncertainty quantification for language model generation with struct

Smoothed Elicitation Complexity for Approximate Gamma-calibration of Discrete Classification Tasks

arXiv:2605.23017v1 Announce Type: new Abstract: One prominent method of evaluating machine learning model trustworthiness is the notion of calibration. In the binary outcome setting, a probabilistic p

StereoGenBench: A Synthetic Multi-Camera Benchmark for Stereo Generation under Controlled Baseline Regimes

arXiv:2605.23237v1 Announce Type: new Abstract: Stereo image and video generation, stereo geometry estimation, and condition-controlled view synthesis require paired data in which the variables that d

AgForce Enables Antigen-conditioned Generative Antibody Design

arXiv:2605.21610v1 Announce Type: new Abstract: Antibody design methods condition on antigen structure to generate complementarity-determining regions (CDR), yet a systematic evaluation of baseline me

Algebraic Machine Learning for Small-to-Medium Datasets Is Competitive against Strong Standard Baselines

arXiv:2605.22155v1 Announce Type: new Abstract: Symbolic methods are generally not considered competitive with strong modern learners on realistic supervised tasks. We evaluate Algebraic Machine Learn

safetyarxiv-cs-lg

Cross-domain benchmarks reveal when coordinated AI agents improve scientific inference from partial evidence

arXiv:2605.22300v1 Announce Type: cross Abstract: Scientific evidence often spans instruments, databases, and disciplines, so no single source records the full phenomenon. This makes it difficult to d

FD-Bench: A Modular and Fair Benchmark for Data-driven Fluid Simulation

arXiv:2505.20349v2 Announce Type: replace-cross Abstract: Data-driven modeling of fluid dynamics has advanced rapidly with neural PDE solvers, yet a fair and strong benchmark remains fragmented due to

Applications

No Epoch Like the Present: Robust Climate Emulation Requires Out-of-Distribution Generalisation

arXiv:2605.22248v1 Announce Type: new Abstract: Climate emulation is an out-of-distribution (OOD) projection task. This is precisely the challenge where modern Machine Learning (ML) methods are most p

applicationsarxiv-cs-lg

VRPRM: Process Reward Modeling via Visual Reasoning

arXiv:2508.03556v3 Announce Type: replace Abstract: Process Reward Model (PRM) is widely used in the post-training of Large Language Model (LLM) because it can perform fine-grained evaluation of the r

AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Grounding

arXiv:2605.22034v1 Announce Type: new Abstract: Visual grounding, the task of localizing objects described by natural-language expressions, is a foundational capability for agricultural AI systems, en

DeepWeb-Bench: A Deep Research Benchmark Demanding Massive Cross-Source Evidence and Long-Horizon Derivation

arXiv:2605.21482v1 Announce Type: new Abstract: Deep research, in which an agent searches the open web, collects evidence, and derives an answer through extended reasoning, is a prominent use case for

Do Vision Models Encode Object-Level Semantic Relatedness? A Cognitive Psychology-Inspired Benchmark

arXiv:1709.03806v2 Announce Type: replace Abstract: Modern vision models have achieved strong object-recognition performance, yet it remains unclear whether their representations encode object-level s

Applications

OpenAI named a Leader in enterprise coding agents by Gartner

OpenAI has been recognized as a Leader in Gartner's 2026 evaluation of enterprise coding agents, highlighting its competitive position in AI-powered code generation and software development tools. Thi

applicationsopenai

Ratchet: A Minimal Hygiene Recipe for Self-Evolving LLM Agents

arXiv:2605.22148v1 Announce Type: cross Abstract: Self-evolving skill libraries, pioneered by Voyager, let frozen LLM agents accumulate reusable knowledge without weight updates, yet recent evaluation

ScenePilot: Controllable Boundary-Driven Critical Scenario Generation for Autonomous Driving

arXiv:2605.21168v1 Announce Type: new Abstract: Safety-critical scenarios are central to evaluating autonomous driving systems, yet their rarity in naturalistic logs makes simulation-based stress test

safetyarxiv-cs-ai

Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding

arXiv:2605.21852v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in general video understanding, their capacity to interpret invo

Structure Retention in Embedding Spaces as a Predictor of Benchmark Performance

arXiv:2605.22202v1 Announce Type: new Abstract: In this paper, we show that high-performing embedding models organize their embedding spaces in a consistent way. We evaluate 25 contemporary embedding

Local Ai

Structured-Sparse Attention for Entity Tracking with Subquadratic Sequence Complexity

arXiv:2605.22476v1 Announce Type: cross Abstract: Entity tracking requires maintaining and updating latent states for entities and attributes over long sequences. Recent task-specific attention operat

local-aiarxiv-cs-cl

Teaching AI Through Benchmark Construction: QuestBench as a Course-Based Practice for Accountable Knowledge Work

arXiv:2605.21413v2 Announce Type: new Abstract: As AI becomes part of everyday learning, many courses teach students to use it mainly as a productivity tool: how to prompt, search, summarize, write, c

AgentAtlas: Beyond Outcome Leaderboards for LLM Agents

arXiv:2605.20530v1 Announce Type: cross Abstract: Large language model agents now act on codebases, browsers, operating systems, calendars, files, and tool ecosystems, but the benchmarks used to evalu

Tutorials

Build an AI-powered recruitment assistant using Amazon Bedrock

In this post, we demonstrate how to build an AI-powered recruitment assistant using Amazon Bedrock that brings efficiencies to candidate evaluation, generates personalized interview questions, and pro

tutorialsaws-ml-blog

Fine-grained Claim-level RAG Benchmark for Law

arXiv:2605.21071v1 Announce Type: new Abstract: The rapid progress of large language models (LLMs) is shifting semantic search toward a question-answering paradigm, where users ask questions and LLMs

Industry

Grok Build 0.1 might be one of the most underestimated AI models right now. We tested it in Kilo Code by asking it to build 5 websites from …

Grok 0.1 was evaluated on its ability to build complete websites from text prompts in a Kilo Code test, demonstrating capabilities that may exceed initial market expectations. The test involved reques

industryelon-musk--x

InternBootcamp Technical Report: Boosting LLM Reasoning with Verifiable Task Scaling

arXiv:2508.08636v2 Announce Type: replace Abstract: Large language models (LLMs) have revolutionized artificial intelligence by enabling complex reasoning capabilities. While recent advancements in re

LAION-C: An Out-of-Distribution Benchmark for Web-Scale Vision Models

arXiv:2506.16950v2 Announce Type: replace Abstract: Out-of-distribution (OOD) robustness is a desired property of computer vision models. Improving model robustness requires high-quality signals from

LASH: Adaptive Semantic Hybridization for Black-Box Jailbreaking of Large Language Models

arXiv:2605.21362v1 Announce Type: new Abstract: Jailbreak attacks expose a persistent gap between the intended safety behavior of aligned large language models and their behavior under adversarial pro

safetyarxiv-cs-cl

Mitigating Label Bias with Interpretable Rubric Embeddings

arXiv:2605.21455v1 Announce Type: new Abstract: Statistical decision algorithms are increasingly deployed in domains where ground-truth labels are hard to obtain, such as hiring, university admissions

safetyarxiv-cs-lg