Search: “evaluation”

EgoSafe: A First-Person Mobile-Captured Benchmark for Visual Safety Understanding

arXiv:2607.26518v1 Announce Type: new Abstract: Reliable visual safety understanding in real-world scenarios demands more than just object recognition; it requires causal reasoning under epistemic unc

30 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Enhancing Automated Machine Learning via Homogeneous Train-Test Splitting Methods

DGX agent

arXiv:2607.26625v1 Announce Type: new Abstract: Accurate model evaluation in machine learning depends critically on how datasets are split into training and testing subsets. Standard random splitting

model-releasesarxiv-cs-lg

30 Jul 2026

Foundation Models for Face Presentation Attack Detection: A Unified Linear-Probing Benchmark

arXiv:2607.26993v1 Announce Type: new Abstract: Face presentation attack detection (PAD) remains challenging under cross-dataset evaluation, where domain shift degrades models trained on a single data

30 Jul 2026

AMPBench-MT: A Homology-Controlled Benchmark for Antimicrobial Peptide Potency, Spectrum, and Safety Prediction

arXiv:2607.25518v1 Announce Type: new Abstract: Computational AMP discovery is often evaluated through AMP/non-AMP recognition, yet follow-up decisions depend on assay-derived evidence such as target-

Benchmarking Deep Learning Models for Raman Spectroscopy Across Open-Source Datasets

arXiv:2601.16107v2 Announce Type: replace Abstract: Deep learning classifiers for Raman spectroscopy are increasingly reported to outperform classical chemometric approaches. However, their evaluation

researcharxiv-cs-lg

Building Large-Scale English-Romanian Literary Translation Resources with Open Models

arXiv:2509.07829v4 Announce Type: replace-cross Abstract: Literary translation has recently gained attention as a distinct and complex task in machine translation research, yet translation by small op

Engine-Equal, Human-Unequal: A Reproducible Outcome Skew in Engine-Assessed Equal Chess Positions

arXiv:2607.25655v1 Announce Type: new Abstract: Among chess opening positions that a strong engine judges essentially equal (Stockfish 18 evaluation within 10 centipawns of zero, depth-stable) and tha

researcharxiv-cs-ai

RoboProcessBench: Benchmarking Process-Aware Understanding in Vision-Language Robotic Manipulation

model-releasesarxiv-cs-ro

arXiv:2606.13040v2 Announce Type: replace Abstract: Vision-language models (VLMs) are increasingly explored as visual critics, reward generators, and failure detectors in robotic manipulation. These r

What’s new in Gemini Enterprise Agent Platform

model-releasesgoogle-cloud-ai

Since we launched Gemini Enterprise Agent Platform a few months ago, we’ve seen inspiring progress from businesses and builders alike. To stir up development, we’ve also shared 13 demos that can walk

Agent-UCT: Upper Confidence Bounds Applied to Trees for Agentic Workflow Optimization with Cost-Awareness

arXiv:2607.24162v1 Announce Type: new Abstract: Optimizing agentic workflows, such as retrieval-augmented generation (RAG) pipelines, requires navigating a combinatorial space of discrete component ch

agentsarxiv-cs-ai

Benchmarking the Domain Gap: Model Selection Instability Under Domain Shift in Video Capsule Endoscopy

arXiv:2607.22736v1 Announce Type: new Abstract: Video capsule endoscopy (VCE) classification is typically evaluated within a single dataset, yet clinical deployment demands robustness across acquisiti

researcharxiv-cs-cv

BioSentinel at EXIST 2026: Soft-Label Optimization with XLM-RoBERTa for Sexism Intent Classification in Memes

arXiv:2607.24137v1 Announce Type: new Abstract: This paper describes the BioSentinel team's participation in EXIST 2026 Task 2.2: Source Intention in Memes, part of the CLEF 2026 evaluation campaign.

researcharxiv-cs-cl

ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding

arXiv:2607.24743v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) hold immense potential to revolutionize clinical practice, yet deploying them in the medical domain is fundam

EditCLEVR: A Paired-Scene Intervention Benchmark for Compositional Faithfulness of Object-Centric Representations

arXiv:2607.22705v1 Announce Type: new Abstract: Object-centric learning aims to represent scenes as objects whose properties can be reused in new combinations. Existing evaluations usually score segme

Gubernaut: A Deterministic Homeostatic Controller for Affect-Regulated LLM Agents, Validated Across Independent Model Families

arXiv:2607.24339v1 Announce Type: new Abstract: Large language model (LLM) agents inherit reactive failure modes: escalation under provocation, sycophantic drift under flattery, perseveration when stu

How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Study

arXiv:2607.10856v2 Announce Type: replace-cross Abstract: The rise of Software Engineering (SE) agents, i.e., LLM-based agents that can understand large codebases and carry out engineering tasks with

agentsarxiv-cs-ai

LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratories

arXiv:2607.23704v1 Announce Type: cross Abstract: The deployment of embodied agents in self-driving laboratories could accelerate scientific discovery, yet their reliability is constrained by the irre

LLM-SoccerArena: Benchmarking LLMs on Real-World Predictions in Sports

arXiv:2607.24573v1 Announce Type: new Abstract: Large language models (LLMs) increasingly support decisions about uncertain future events, yet evaluating their ability to forecast real-world outcomes

LU-500: A Logo Benchmark for Concept Unlearning

arXiv:2607.24101v1 Announce Type: cross Abstract: Concept unlearning is increasingly used to limit the reproduction of protected or unsafe visual concepts in text-to-image models. Existing evaluations

Task-Conditional Faithfulness Auditing of Multimodal LLMs for Grid Diagnosis

arXiv:2607.24539v1 Announce Type: new Abstract: Multimodal large language models (LLMs) can combine topology, measurements, and incident text for grid diagnosis, yet answer accuracy does not establish

researcharxiv-cs-ai

Transfer Learning Architectures for Scalable Multi-Fidelity Bayesian Optimization

arXiv:2607.23404v1 Announce Type: new Abstract: Self-driving laboratories increasingly rely on multi-fidelity Bayesian optimization (MFBO) to balance cheap, approximate evaluations against scarce, exp

IR275K: A Benchmark for Infrared Multi-Frame Super-Resolution Toward Efficient Remote Sensing

arXiv:2607.22380v1 Announce Type: new Abstract: Efficient processing is becoming increasingly important in infrared remote sensing, where satellite constellations produce large volumes of observations

27 Jul 2026

Ha! It did it: 'We introduce BenchBenchBenchBenchBench (BBBBB), an executable benchmark of AI-authored conformance suites for benchmark-eval…

model-releasesethan-mollick--x

Ha! It did it: 'We introduce BenchBenchBenchBenchBench (BBBBB), an executable benchmark of AI-authored conformance suites for benchmark-evaluation metrics' I really thought it would treat 'now do benc

25 Jul 2026

Local Ai

Auditing Evidence Use in Medical LLM Diagnosis

arXiv:2607.20848v1 Announce Type: new Abstract: Medical LLMs are often evaluated by whether they select the correct diagnosis, but diagnostic accuracy alone does not show whether the model used the ca

local-aiarxiv-cs-ai

CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA

arXiv:2607.21155v1 Announce Type: cross Abstract: Knowledge-Intensive Visual Question Answering (KI-VQA) benchmarks evaluate Vision-Language Models (VLMs) as multimodal knowledge assistants by requiri

DataPrep-Bench: Benchmarking LLMs as Training Data Preparators

arXiv:2607.20465v1 Announce Type: cross Abstract: The quality of training data fundamentally determines the capabilities of large language models (LLMs), yet no unified benchmark exists to measure how

Safety

HERMES: Heterogeneous Edge-Relational Multi-Head Embedded SSM Attention for Traffic Conflict Prediction at Signalized Intersections

arXiv:2607.20505v1 Announce Type: cross Abstract: Surrogate safety measures (SSMs) enable proactive traffic safety assessment, but many existing methods evaluate pairwise interactions independently or

safetyarxiv-cs-lg

StabilityBench: Benchmarking Instability in LLMs

arXiv:2607.20558v1 Announce Type: cross Abstract: AI Assistants are increasingly deployed in high-stakes settings, such as healthcare or government services. Yet their real-world behavior remains poor

Telco-GAIA: Bilingual Benchmark for Agents in Telecom Domain

arXiv:2607.20510v1 Announce Type: new Abstract: We introduce Telco-GAIA, a bilingual, multi-modal benchmark for evaluating tool-using agents on the data of a real-world telecommunications operator. Te

Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction

arXiv:2607.20911v1 Announce Type: new Abstract: We introduce Tencent WorkBuddy Bench, a multi-domain evaluation suite for coding agents; this report documents its construction methodology, scoring pro

ViSTR-Bench: Can MLLMs Reason from Continuous Visual Cues in Dynamic Scenes?

arXiv:2607.20868v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable success across diverse expert-level tasks, but they still struggle with fundamental ab

D2VBench: Benchmarking Large Language Models with Value Dilemmas in Daily Scenarios

arXiv:2607.19834v1 Announce Type: new Abstract: With the wide application of large language models (LLMs) in real-world scenarios, the value implication of their outputs is crucial. However, existing

23 Jul 2026

Safety

Stress Testing Concept Erasure with Large Language Model Agents

arXiv:2607.17890v2 Announce Type: replace Abstract: Concept erasure aims to remove semantic concepts from a trained generative model and is increasingly important for responsible AI deployment. Howeve

safetyarxiv-cs-ai

23 Jul 2026

Beyond Description: Cognitively Benchmarking Fine-Grained Action for Embodied Agents

arXiv:2511.18685v4 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) show promising results as decision-making engines for embodied agents operating in complex, physical enviro

DevicesWorld: Benchmarking Cross-Device Agents in Heterogeneous Environments

arXiv:2607.13465v1 Announce Type: cross Abstract: LLM-based agents have rapidly improved at operating individual digital environments such as mobile applications, desktop systems, and smart homes. How

EgoProceVQA: A Novel Egocentric Procedural Understanding Task with Self-Skill-Exploration Agent

arXiv:2607.13792v1 Announce Type: new Abstract: Most daily activities are inherently procedural. However, existing evaluations for egocentric video understanding seldom address procedural understandin

HRIBench: Benchmarking Interaction-Centric Human-Robot Collaboration

arXiv:2607.13056v1 Announce Type: cross Abstract: Current vision-language-action (VLA) benchmarks primarily evaluate isolated manipulation skills while leaving human-robot interaction structure largel

NeMo: Needle in a Montage for Video-Language Understanding

arXiv:2509.24563v3 Announce Type: replace Abstract: Recent advances in video large language models (VideoLLMs) call for new evaluation protocols and benchmarks for video-language understanding. Inspir

Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation

arXiv:2607.14006v1 Announce Type: cross Abstract: Penetration testing traditionally evaluates whether adversaries can exploit weaknesses in software, infrastructure, configurations, or operational con

agentsarxiv-cs-ai

Designing Agent-Ready Websites for AI Web Agents: A Framework for Machine Readability, Actionability, and Decision Reliability

arXiv:2607.12056v1 Announce Type: new Abstract: Online shopping is increasingly shifting toward a model in which AI agents independently search for products, compare options, evaluate constraints, and

FairCoder: Probing LLM Bias in High-Stakes Decision Making via Coding Tasks

arXiv:2501.05396v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used in high-stakes decisions such as hiring and college admissions, making their social bias a critic

Safety

From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness

arXiv:2607.12166v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are the standard for decomposing superposed neural representations into interpretable features, and evaluation relies predomi

safetyarxiv-cs-lg

Image Matching Filtering and Refinement by Planes and Beyond

arXiv:2411.09484v5 Announce Type: replace Abstract: This paper provides a consistent and extensive evaluation of state-of-the-art filtering and refinement methods on common image matching pipelines. U

MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations

arXiv:2607.12893v1 Announce Type: new Abstract: Long-term memory has become a foundational capability for LLM-based agents that accompany users across extended, multi-session interactions. Existing be

NEW paper from Microsoft and colleagues. Debugging agent trajectories at scale is challenging. This is a clever approach to monitor and impr…

NEW paper from Microsoft and colleagues. Debugging agent trajectories at scale is challenging. This is a clever approach to monitor and improve agents in production. The problem: Finding which step in

agentsdair-ai--x

Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence

arXiv:2607.12477v1 Announce Type: new Abstract: Autonomous UAV systems increasingly rely on multimodal large language models (MLLMs) to operate in complex real-world environments. Such embodied scenar

WANDR tests how well agents discover large sets of entities and verify specific facts about each one. It provides a dense, interpretable eva…

WANDR tests how well agents discover large sets of entities and verify specific facts about each one. It provides a dense, interpretable eval signal that reveals whether an agent fails, and where. The

agentsperplexity--x

14 Jul 2026

ICDAR 2026 HIPE-OCRepair Competition on LLM-Assisted OCR Post-Correction for Historical Documents

arXiv:2607.08143v1 Announce Type: cross Abstract: We present the results of HIPE-OCRepair-2026, an ICDAR competition on LLM-assisted OCR post-correction of historical documents. OCR post-correction re