Search: “evaluation”

Bounded Agents: Delegation Security for Multi-Agent AI Systems

arXiv:2608.15888v1 Announce Type: new Abstract: LLM-based agents can act on behalf of a user to access cloud services, call tools, or invoke agents. At session start, the agent's permissions are set b

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

CFR without Unbiasedness: Deterministic Guarantees for Persistent Public-Chance Schedules

DGX agent

arXiv:2608.14761v1 Announce Type: cross Abstract: At a finite public-chance cut, counterfactual regret minimization (CFR) must choose how many outcomes to evaluate before each regret update. Exact eva

model-releasesarxiv-cs-lg

18 Aug 2026

DeepInsight II: One Trace from Benchmark to Robot

arXiv:2608.16556v1 Announce Type: new Abstract: Across a Physical AI stack, evaluation maturity is inversely aligned with deployment risk: foundation models enjoy mature, standardized harnesses, while

Dense Expands, Sparse Anchors: Channel-Asymmetric Query Expansion for Hybrid Retrieval

arXiv:2608.15851v1 Announce Type: cross Abstract: LLM-based query expansion improves retrieval by generating document-like passages. In hybrid retrieval, however, most evaluations fuse fixed top-L den

DriveCache: Action-Aware Caching for Driving World Model Inference

arXiv:2608.16354v1 Announce Type: new Abstract: Driving video generation models support autonomous-driving development by predicting controllable future scenes for simulation, planning evaluation, and

Foresight-England: Development of a National-Scale Generative AI Model of Electronic Health Records for Medical Event Prediction across the COVID-19 Pandemic

arXiv:2608.16273v1 Announce Type: cross Abstract: Foresight-England (Foresight-E) is the first national-scale generative foundation model of electronic health records (EHRs), developed as a research p

Research

Generation of Synthetic Fingerphotos with GANs

arXiv:2608.15029v1 Announce Type: new Abstract: Contactless fingerprinting is an emerging approach to biometric authentication that allows users to scan their fingerprints without touching a scanner.

researcharxiv-cs-cv

Grounding Healthcare LLMs in a Causal Knowledge Graph: Framework, Metrics, and a Cardiovascular Pilot

arXiv:2608.15382v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly proposed for healthcare decision support, but their evaluations still reward single-answer accuracy rather

applicationsarxiv-cs-ai

Grounding Robot Generalization in Training Data via Retrieval-Augmented VLMs

model-releasesarxiv-cs-ro

arXiv:2603.11426v3 Announce Type: replace Abstract: Recent work on robot manipulation has advanced policy generalization to novel scenarios. However, it is often difficult to characterize how differen

PLSQLBench: Benchmarking LLM Systems for Executable Procedural Database Programming

arXiv:2608.15931v1 Announce Type: new Abstract: We present PLSQLBench, to our knowledge the first benchmark for evaluating whether LLMs can write executable PL/SQL programs, with correctness measured

Revisiting the Performance of Generative Artificial Intelligence on Introductory Object-Oriented Programming Assessments: Insights from 2026

arXiv:2608.16318v1 Announce Type: cross Abstract: Recent advances in Generative Artificial Intelligence (GenAI) have substantially improved the ability of large language models (LLMs) to generate and

VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End?

arXiv:2608.15265v1 Announce Type: new Abstract: Constructing an interactive 3D open world from a user query is important. However, existing methods are primarily evaluated on idealized, simple queries

VideoGAIA: A Benchmark for General AI Assistants on Agentic Video Understanding

arXiv:2608.14718v1 Announce Type: cross Abstract: Video understanding is a fundamental task for evaluating the capabilities of multimodal large language models (MLLMs). However, existing leading model

What to Forget in Unlearning? Forget Set Curation for Language Models

arXiv:2608.14855v1 Announce Type: new Abstract: Machine unlearning aims to remove targeted data or behaviors from a trained model without retraining from scratch. Yet most evaluations assume that the

AI Research Preference Models

arXiv:2608.13940v1 Announce Type: new Abstract: AI research agents (AIRA) can now propose, implement, and evaluate their own machine learning experiments, but progress on frontier tasks is throttled b

AlignFace: Human-Aligned Face Similarity Metric with Interpretable Concept Relations

arXiv:2608.14130v1 Announce Type: cross Abstract: Computer vision models for generated facial content, such as face editing and privacy protection, increasingly affect people, requiring similarity met

safetyarxiv-cs-ai

CPI-Bench: A Comprehensive,Practical and Intelligent Benchmark for Real-World Image Editing

model-releasesarxiv-cs-cv

arXiv:2608.14546v1 Announce Type: new Abstract: With the rapid advancement of image editing models and their widespread application across various domains, there is an increasingly urgent need to depl

LLM-Advisor: An LLM Advisor for Cost-efficient Path Planning across Multiple Terrains

arXiv:2503.01236v3 Announce Type: replace-cross Abstract: This paper addresses fixed-graph terrain-aware path refinement, in which a global planner is restricted to a predefined route space and may re

Measuring Cross-Task Behavioral Consistency in Language Model Agents

arXiv:2608.13598v1 Announce Type: new Abstract: Agent evaluation relies almost entirely on outcome metrics such as success rate, which capture whether an agent succeeds but not how consistently it beh

Measuring Fairness in Large Audio Language Models via Semantic-Aware Bias Estimation

arXiv:2608.13624v1 Announce Type: cross Abstract: Large Audio Language Models (LALMs) have seen increasing use for audio understanding tasks such as speech recognition and audio question answering, ra

safetyarxiv-cs-ai

PACE-Bench: Benchmarking Physics Adaptation via Code Evolution in Dynamic Environments

arXiv:2608.14441v1 Announce Type: new Abstract: Self-evolving agents improve future behavior from interaction experience, yet existing evaluations typically optimize under fixed execution conditions a

Recent Advances in Deep Learning-Based Drug-Target Binding Affinity Prediction

model-releasesarxiv-cs-lg

arXiv:2608.13797v1 Announce Type: new Abstract: Computational approaches to drug discovery involve multiple sub-problems, and among them, drug-target binding affinity prediction plays an important rol

A Browser-Native Digital Test Range for Benchmarking 4D Ocean-Glider Planning Algorithms

arXiv:2608.13511v1 Announce Type: new Abstract: Repeated in-situ evaluation of ocean-glider planners requires scarce vehicles, operators, deployment and recovery resources, and ocean conditions that c

safetyarxiv-cs-ro

Don't Want Your LLM to Recommend Nuclear Strike? Try Asking It in Japanese

arXiv:2608.12373v1 Announce Type: new Abstract: Large language models are increasingly used in strategic and advisory contexts, yet their safety alignment is typically evaluated in English only. We te

H-VAEP and H-xT: Valuing Offensive On-the-Ball Actions in Handball by Estimating Probabilities

arXiv:2608.12926v1 Announce Type: cross Abstract: Traditional player evaluation in professional handball relies on basic box-score metrics or heuristic indices, which fail to credit the multi-player b

applicationsarxiv-cs-ai

How Good are Foundation Models in Longitudinal MRI Disease Progression Reasoning?

model-releasesarxiv-cs-cv

arXiv:2608.13309v1 Announce Type: new Abstract: Magnetic Resonance Imaging (MRI) interpretation is fundamental to clinical decision-making, requiring radiologists to integrate multi-view anatomical pl

Large Language Models Pass the History Exam But Miss the <<History>>: A Polish High School Exit Exam Matura Benchmark

arXiv:2608.12343v1 Announce Type: new Abstract: AI chatbots are widely used by students as knowledge sources, yet LLM benchmarks rarely assess interpretative historical reasoning. We evaluate eight le

Learning Under Treatment-Induced Label Indeterminacy with Expert Annotations of Counterfactual Outcomes: A Case Study in Neurological Prognostication

arXiv:2608.12477v1 Announce Type: new Abstract: Clinical prediction models are often developed as if the outcome of interest were cleanly observed for every patient. This assumption fails when treatme

safetyarxiv-cs-lg

LigBench: A Unified and Human-Aligned Benchmark for LLM-based Research Idea Generation

arXiv:2608.13136v1 Announce Type: cross Abstract: With the rapid advancement of large language models (LLMs), research idea generation has attracted increasing attention. Existing approaches enable LL

NARU: A Benchmark for NARrative Evolution and Cultural Nuance Understanding in Japanese Extreme Long Video

arXiv:2608.13210v1 Announce Type: cross Abstract: Long-form video understanding encompasses tasks that go beyond retrieving isolated events, including tracking an evolving narrative and interpreting s

On Measuring Semantic Preservation in Legal Ontology Learning

arXiv:2608.12326v1 Announce Type: new Abstract: Ontology learning transforms unstructured text into structured representations for automated reasoning. Yet structuring information risks losing it, and

applicationsarxiv-cs-cl

SLAyiNG: A Diverse and Community-validated Dataset of Queer Slang

arXiv:2509.17449v2 Announce Type: replace Abstract: Queer vernacular is rarely studied in NLP, despite advancements in resources and evaluation for other sociolects and informal language. Because of t

StarEmbed: Benchmarking Time Series Foundation Models on Astronomical Observations of Variable Stars

arXiv:2510.06200v4 Announce Type: replace-cross Abstract: Current time series foundation model (TSFM) training corpora largely omit data with certain complexities like irregular temporal sampling. Ast

The 'Knowledge-Behavior Gap' in Cultural Taboo Safety of Large Language Models

arXiv:2608.12341v1 Announce Type: new Abstract: Cultural taboo safety is essential for deploying large language models (LLMs), as culturally insensitive outputs may cause offense or even social harm.

TsuGO: Probing Search Efficiency in LLM Reasoning via Go Life-and-Death Problems

arXiv:2608.13221v1 Announce Type: new Abstract: The evaluation of LLM reasoning is moving from final-answer accuracy to process-level assessment, yet existing methods still fail to capture how models

Vero: Can AI Agents Build Formally Verified Software Repositories?

arXiv:2608.13522v1 Announce Type: cross Abstract: AI agents are increasingly used for programming, but do not provide any guarantee on the correctness of generated code. Verified code generation, in w

A method for tissue-mask supported whole-body image registration in the UK Biobank

arXiv:2512.02702v3 Announce Type: replace Abstract: The UK Biobank is a large-scale study collecting imaging and non-imaging health data. Robust and accurate inter-subject image registration of these

safetyarxiv-cs-cv

Backtrader-Bench: Benchmarking LLM Agents on Algorithmic Trading with Self-Generated MCQs

arXiv:2608.11232v1 Announce Type: cross Abstract: Evaluating LLM coding agents in algorithmic trading is difficult because static benchmarks risk data contamination and numerical backtest outputs requ

Beyond Single-Turn Confidence: Trajectory-Adapted Uncertainty Quantification for LLM Agents

arXiv:2608.11552v1 Announce Type: cross Abstract: Uncertainty quantification (UQ) methods for language models are typically evaluated on single-turn outputs, where uncertainty is attached to one gener

Trust Region Constrained Bayesian Optimization with Penalized Constraint Handling

arXiv:2603.24567v2 Announce Type: replace-cross Abstract: Constrained optimization in high-dimensional black-box settings is difficult due to expensive evaluations, the lack of gradient information, a

applicationsarxiv-cs-lg

Unmasking Toxic Mimicry in Medical Offline Reinforcement Learning for ICU Sepsis Management via Counterfactual Clinical Audits

arXiv:2608.11410v1 Announce Type: new Abstract: Offline reinforcement learning (RL) offers considerable promise for optimizing ICU treatment decisions, yet standard evaluation metrics Mean Squared Err

safetyarxiv-cs-lg

VICBench: A Multi-Language Benchmark for Code Vulnerability Detection

arXiv:2608.12246v1 Announce Type: cross Abstract: Evaluating security vulnerability detection tools requires benchmark datasets with vulnerability-inducing commits (VICs) - the commits that first intr

Who Would You Vote For? Auditing Political Alignment in LLMs: An Italian Case-Study

arXiv:2608.11649v1 Announce Type: new Abstract: As users increasingly turn to Large Language Models (LLMs) for information and advice on political matters, particularly during election periods, the po

A HamNoSys-Guided Dataset and Baselines for Fine-Grained Isolated Handshape Recognition in Sign Language

arXiv:2608.10588v1 Announce Type: cross Abstract: Purpose: Fine-grained handshape recognition supports computational sign-language transcription, recognition, and translation, but broad, phonetically

Actions Speak Louder than Words: Measuring Cross-Lingual Policy Retention in Tool-Using Agents

arXiv:2608.11110v1 Announce Type: new Abstract: When a tool-using agent is given the same task in a different language, does it still take the same steps? Multilingual evaluation rarely asks: it compa

DashArena: Benchmarking LLMs on Interactive Analytic Dashboard Generation

arXiv:2608.10567v1 Announce Type: new Abstract: Analytic dashboards combine coordinated views and interactions for data exploration and decision-making. Recent models can generate them from data and n

Research

Decomposition-Induced Context-Memory Conflict: When Fact-Checking Pipelines Contradict Their Own Source Text

arXiv:2608.10627v1 Announce Type: new Abstract: Decompose-then-verify pipelines, including FActScore-style fact-checkers and long-form factuality evaluators, first split a passage into atomic claims b

researcharxiv-cs-cl

DuplexWorld: Can voice agents help you get through the day?

arXiv:2608.10716v1 Announce Type: cross Abstract: Speech-to-speech (S2S) voice agents are increasingly being incorporated into enterprise for customer care and as daily companions for consumers owing