Search: “evaluation”

Quantum Machine Learning for Colorectal Cancer Data: Anastomotic Leak Classification and Risk Factors

arXiv:2604.13951v1 Announce Type: new Abstract: This study evaluates colorectal risk factors and compares classical models against Quantum Neural Networks (QNNs) for anastomotic leak prediction. Analy

researcharxiv-cs-lg

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

RiskWebWorld: A Realistic Interactive Benchmark for GUI Agents in E-commerce Risk Management

DGX agent

arXiv:2604.13531v1 Announce Type: cross Abstract: Graphical User Interface (GUI) agents show strong capabilities for automating web tasks, but existing interactive benchmarks primarily target benign,

model-releasesarxiv-cs-lg

16 Apr 2026

ROBOGATE: Adaptive Failure Discovery for Safe Robot Policy Deployment via Two-Stage Boundary-Focused Sampling

model-releasesarxiv-cs-ro

arXiv:2603.22126v3 Announce Type: replace Abstract: Deploying learned robot manipulation policies in industrial settings requires rigorous pre-deployment validation, yet exhaustive testing across high

SFT-GRPO Data Overlap as a Post-Training Hyperparameter for Autoformalization

arXiv:2604.13515v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) followed by Group Relative Policy Optimization (GRPO) is a common post-training recipe. We conduct a controlled ablation ov

safetyarxiv-cs-lg

Singularity Avoidance in Inverse Kinematics: A Unified Treatment of Classical and Learning-based Methods

arXiv:2604.13405v1 Announce Type: new Abstract: Singular configurations cause loss of task-space mobility, unbounded joint velocities, and solver divergence in inverse kinematics (IK) for serial manip

researcharxiv-cs-ro

Target-Bench: Can Video World Models Achieve Mapless Path Planning with Semantic Targets?

arXiv:2511.17792v2 Announce Type: replace Abstract: While recent video world models can generate highly realistic videos, their ability to perform semantic reasoning and planning remains unclear and u

5/5 The takeaway: If your agent relies on an LLM judge for selection accuracy, measuring code quality isn’t enough; you need a measure of th…

5/5 The takeaway: If your agent relies on an LLM judge for selection accuracy, measuring code quality isn’t enough; you need a measure of the model's inductive bias toward the 'fingerprint' of a gold

safetyai21-labs--x

A Large-Scale Comparative Analysis of Imputation Methods for Single-Cell RNA Sequencing Data

model-releasesarxiv-cs-lg

arXiv:2603.24626v2 Announce Type: replace-cross Abstract: Background: Single-cell RNA sequencing (scRNA-seq) enables gene expression profiling at cellular resolution but is inherently affected by spar

Industry

Adobe unveils Firefly AI Assistant, which can orchestrate and execute multistep tasks across Creative Cloud apps, available in public beta in the coming weeks (Ivan Mehta/TechCrunch)

Ivan Mehta / TechCrunch: Adobe unveils Firefly AI Assistant, which can orchestrate and execute multistep tasks across Creative Cloud apps, available in public beta in the coming weeks — Last October,

industrytechmeme

Are Video Reasoning Models Ready to Go Outside?

arXiv:2603.10652v2 Announce Type: replace-cross Abstract: In real-world deployment, vision-language models often encounter disturbances such as weather, occlusion, and camera motion. Under such condit

Broadening the Applicability of Conditional Syntax Splitting for Reasoning from Conditional Belief Bases

arXiv:2604.12660v1 Announce Type: new Abstract: In nonmonotonic reasoning from conditional belief bases, an inference operator satisfying syntax splitting postulates allows for taking only the relevan

Calibrated Confidence Estimation for Tabular Question Answering

arXiv:2604.12491v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed for tabular question answering, yet calibration on structured data is largely unstudied. This pap

researcharxiv-cs-cl

CodeSpecBench: Benchmarking LLMs for Executable Behavioral Specification Generation

arXiv:2604.12268v1 Announce Type: cross Abstract: Large language models (LLMs) can generate code from natural language, but the extent to which they capture intended program behavior remains unclear.

CODESTRUCT: Code Agents over Structured Action Spaces

arXiv:2604.05407v2 Announce Type: replace Abstract: LLM-based code agents treat repositories as unstructured text, applying edits through brittle string matching that frequently fails due to formattin

Coding-Free and Privacy-Preserving MCP Framework for Clinical Agentic Research Intelligence System

arXiv:2604.12258v1 Announce Type: cross Abstract: Clinical research involves labor-intensive processes such as study design, cohort construction, model development, and documentation, requiring domain

agentsarxiv-cs-ai

CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systems

arXiv:2604.12312v1 Announce Type: new Abstract: As Large Language Models (LLMs) are increasingly deployed as task-oriented agents in enterprise environments, ensuring their strict adherence to complex

Designing Reliable LLM-Assisted Rubric Scoring for Constructed Responses: Evidence from Physics Exams

arXiv:2604.12227v1 Announce Type: new Abstract: Student responses in STEM assessments are often handwritten and combine symbolic expressions, calculations, and diagrams, creating substantial variation

safetyarxiv-cs-ai

From Imitation to Discrimination: Progressive Curriculum Learning for Robust Web Navigation

arXiv:2604.12666v1 Announce Type: cross Abstract: Text-based web agents offer computational efficiency for autonomous web navigation, yet developing robust agents remains challenging due to the noisy

IAD-Unify: A Region-Grounded Unified Model for Industrial Anomaly Segmentation, Understanding, and Generation

arXiv:2604.12440v1 Announce Type: cross Abstract: Real-world industrial inspection requires not only localizing defects, but also explaining them in natural language and generating controlled defect e

Inside VAKRA: Reasoning, Tool Use, and Failure Modes of Agents

VAKRA (Versatile Agent and Knowledge Reasoning Assessment) is a benchmark developed by IBM Research to evaluate AI agents on complex reasoning, tool use, and multi-step task execution across diverse d

agentshugging-face

Industry

Just asked ChatGPT if i'm smart

A Reddit post from the r/ChatGPT community in which a user shares their experience of asking ChatGPT to evaluate their intelligence, likely showcasing the AI's response. The post probably highlights h

industryr-chatgpt

Let's talk parsing tables. Two days ago we launched ParseBench,the first document OCR benchmark built for AI agents. This deep dive breaks d…

model-releasesjerry-liu--x

Let's talk parsing tables. Two days ago we launched ParseBench,the first document OCR benchmark built for AI agents. This deep dive breaks down TableRecordMatch (GTRM), our metric for evaluating compl

Machine Learning-Based Real-Time Detection of Compensatory Trunk Movements Using Trunk-Wrist Inertial Measurement Units

arXiv:2604.12591v1 Announce Type: new Abstract: Compensatory trunk movements (CTMs) are commonly observed after stroke and can lead to maladaptive movement patterns, limiting targeted training of affe

researcharxiv-cs-ro

MedConcept: Unsupervised Concept Discovery for Interpretability in Medical VLMs

arXiv:2604.11868v1 Announce Type: new Abstract: While medical Vision-Language models (VLMs) achieve strong performance on tasks such as tumor or organ segmentation and diagnosis prediction, their opaq

M^star: Every Task Deserves Its Own Memory Harness

arXiv:2604.11811v1 Announce Type: cross Abstract: Large language model agents rely on specialized memory systems to accumulate and reuse knowledge during extended interactions. Recent architectures ty

agentsarxiv-cs-ai

MVAdapt: Zero-Shot Multi-Vehicle Adaptation for End-to-End Autonomous Driving

arXiv:2604.11854v1 Announce Type: cross Abstract: End-to-End (E2E) autonomous driving models are usually trained and evaluated with a fixed ego-vehicle, even though their driving policy is implicitly

Local Ai

Ollama Open-Source Agent Self-Reflection Harness

An open-source agent self-reflection harness built on top of Ollama, shared in the r/ollama community, that enables locally run LLMs to evaluate and iteratively refine their own outputs. The project p

local-air-ollama

One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness

arXiv:2604.13006v1 Announce Type: cross Abstract: Instruction-tuned large language models produce helpful, structured responses, but how robust is this helpfulness when trivially constrained? We show

ReasonXL: Shifting LLM Reasoning Language Without Sacrificing Performance

arXiv:2604.12378v1 Announce Type: new Abstract: Despite advances in multilingual capabilities, most large language models (LLMs) remain English-centric in their training and, crucially, in their produ

StoryScope: Investigating idiosyncrasies in AI fiction

arXiv:2604.03136v4 Announce Type: replace Abstract: As AI-generated fiction becomes increasingly prevalent, questions of authorship and originality are becoming central to how written work is evaluate

T2I-BiasBench: A Multi-Metric Framework for Auditing Demographic and Cultural Bias in Text-to-Image Models

arXiv:2604.12481v1 Announce Type: new Abstract: Text-to-image (T2I) generative models achieve impressive visual fidelity but inherit and amplify demographic imbalances and cultural biases embedded in

The Effect of Document Selection on Query-focused Text Analysis

arXiv:2604.12099v1 Announce Type: cross Abstract: Analyses of document collections often require selecting what data to analyze, as not all documents are relevant to a particular research question and

researcharxiv-cs-cl

The Verification Tax: Fundamental Limits of AI Auditing in the Rare-Error Regime

model-releasesarxiv-cs-lg

arXiv:2604.12951v1 Announce Type: new Abstract: The most cited calibration result in deep learning -- post-temperature-scaling ECE of 0.012 on CIFAR-100 (Guo et al., 2017) -- is below the statistical

Local Ai

Time-reversed Flow Matching with Worst Transport in High-dimensional Latent Space for Image Anomaly Detection

arXiv:2508.05461v3 Announce Type: replace Abstract: Likelihood-based deep generative models have been widely investigated for Image Anomaly Detection (IAD), particularly Normalizing Flows, yet their s

local-aiarxiv-cs-cv

Universal NER v2: Towards a Massively Multilingual Named Entity Recognition Benchmark

arXiv:2604.12744v1 Announce Type: new Abstract: While multilingual language models promise to bring the benefits of LLMs to speakers of many languages, gold-standard evaluation benchmarks in most lang

WebChain: A Large-Scale Human-Annotated Dataset of Real-World Web Interaction Traces

arXiv:2603.05295v3 Announce Type: replace Abstract: We introduce WebChain, the largest open-source dataset of human-annotated trajectories on real-world websites, designed to accelerate reproducible r

safetyarxiv-cs-ai

What is the criteria for a ML paper to be published?[D]

researchr-machinelearning

This r/MachineLearning discussion thread explores the standards and expectations reviewers and program committees use when evaluating ML papers for acceptance at top venues such as NeurIPS, ICML, and

Active Inference with a Self-Prior in the Mirror-Mark Task

arXiv:2604.09673v1 Announce Type: cross Abstract: The mirror self-recognition test evaluates whether a subject touches a mark on its own body that is visible only in a mirror, and is widely used as an

Hardware

AIRA_2: Overcoming Bottlenecks in AI Research Agents

arXiv:2603.26499v2 Announce Type: replace Abstract: Existing research has identified three structural performance bottlenecks in AI research agents: (1) synchronous single-GPU execution constrains sam

hardwarearxiv-cs-ai

An Iterative Utility Judgment Framework Inspired by Philosophical Relevance via LLMs

arXiv:2406.11290v3 Announce Type: replace-cross Abstract: Relevance and utility are two frequently used measures to evaluate the effectiveness of an information retrieval (IR) system. Relevance emphas

Benchmarking Vision-Language Models under Contradictory Virtual Content Attacks in Augmented Reality

arXiv:2604.05510v2 Announce Type: replace Abstract: Augmented reality (AR) has rapidly expanded over the past decade. As AR becomes increasingly integrated into daily life, its security and reliabilit

Budget-Aware Uncertainty for Radiotherapy Segmentation QA Using nnU-Net

arXiv:2604.11798v1 Announce Type: cross Abstract: Accurate delineation of the Clinical Target Volume (CTV) is essential for radiotherapy planning, yet remains time-consuming and difficult to assess, e

safetyarxiv-cs-ai

CLAY: Conditional Visual Similarity Modulation in Vision-Language Embedding Space

arXiv:2604.11539v1 Announce Type: cross Abstract: Human perception of visual similarity is inherently adaptive and subjective, depending on the users' interests and focus. However, most image retrieva

Comparative Analysis of Large Language Models in Healthcare

arXiv:2604.10316v1 Announce Type: new Abstract: Background: Large Language Models (LLMs) are transforming artificial intelligence applications in healthcare due to their ability to understand, generat

Counting to Four is still a Chore for VLMs

arXiv:2604.10039v1 Announce Type: new Abstract: Vision--language models (VLMs) have achieved impressive performance on complex multimodal reasoning tasks, yet they still fail on simple grounding skill

Credit-Budgeted ICPC-Style Coding: When Agents Must Pay for Every Decision

arXiv:2604.10182v1 Announce Type: new Abstract: Current evaluations of autonomous coding agents assume an unrealistic, infinite-resource environment. However, real-world software engineering is a reso

agentsarxiv-cs-ai

Cybersecurity Looks Like Proof of Work Now

model-releasessimon-willison

Cybersecurity Looks Like Proof of Work Now The UK's AI Safety Institute recently published Our evaluation of Claude Mythos Preview’s cyber capabilities, their own independent analysis of Claude Mythos

Diagnosing Failure Modes of Neural Operators Across Diverse PDE Families

arXiv:2601.11428v4 Announce Type: replace Abstract: Neural PDE solvers have shown strong performance on standard benchmarks, but their robustness under deployment-relevant distribution shifts remains

researcharxiv-cs-lg