Search: “evaluation”

Toward Personal Intelligence Through Cooperative Observation

arXiv:2608.17128v1 Announce Type: new Abstract: A personal AI system needs a model of the user's goals, constraints, and ongoing commitments to plan and act on their behalf, and the quality of that mo

researcharxiv-cs-ai

19 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Toward the Optimal Regret-Instability Trade-off in Multi-Armed Bandits

DGX agent

arXiv:2608.17841v1 Announce Type: cross Abstract: Multi-armed bandit algorithms are evaluated by regret, yet comparable regret can coexist with different allocations across independent runs. We study

researcharxiv-cs-lg

19 Aug 2026

TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation

arXiv:2608.17588v1 Announce Type: new Abstract: Agent Skills package reusable natural language procedures with executable resources, enabling software agents to acquire task specific capabilities with

19 Aug 2026

Wuying-Browser-Agent: Real-World Centric Fundamental Long-Horizon Browser Agents

arXiv:2608.17319v1 Announce Type: new Abstract: Browser agents perform well on short, clean demonstrations, but real deployment is fundamentally different: agents must sustain dozens of decisions on l

19 Aug 2026

A Human-LLM Teaming Framework for Privacy Risk Analysis: An Illustration with CBDC-Based Welfare Schemes

arXiv:2608.16461v1 Announce Type: cross Abstract: Central Bank Digital Currency (CBDC)-based welfare schemes may be potentially privacy invasive as they process significant volumes of beneficiary pers

researcharxiv-cs-ai

AnchorScore: A CLIP-Based Diagnostic of MLLM Annotation Difficulty

arXiv:2608.16690v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are widely used for automated annotation, yet their per-class accuracy varies widely (e.g., 12%-98% across the

researcharxiv-cs-cv

Automatic or Controlled? Repetition Priming Reveals Divergent Processing in Base LLMs, Instruct LLMs, and Humans

arXiv:2608.14681v1 Announce Type: cross Abstract: Words recur constantly in natural language use, yet it remains unclear whether language models reactivate prior representations or re-evaluate repeate

Beat the Counter First: A Baseline for Temporal-Graph Anomaly Detectors

arXiv:2608.15965v1 Announce Type: new Abstract: Progress in streaming, edge-level graph anomaly detection (GAD) has been marked by increasingly elaborate architectures, from count-min-sketch chi squar

Benchmarking Frontier Text-to-Image Models on Image-Description Prompts

model-releasesarxiv-cs-cv

arXiv:2608.14976v1 Announce Type: new Abstract: Text-to-image models are typically reported on average-case prompts, which understates the gap between systems on compositionally demanding requests inv

CAPO: Constraint-Aware Prompt Optimization for LLM Agents

arXiv:2608.16068v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed as agents that rely on system prompts to use tools and complete tasks. Such deployments impose

Agents

Dear Algo: A Precision-First Agentic Intent Layer for Unified Search and Recommendation

arXiv:2608.15877v1 Announce Type: new Abstract: Search and recommendation serve a shared discovery objective but encode intent differently. We study this boundary through Dear Algo on Threads, a deplo

agentsarxiv-cs-ai

DeepOHeat-v2: Self-Improving Operator Learning for Fast and Trustworthy Thermal Optimization in 3D-IC Design

arXiv:2608.16080v1 Announce Type: new Abstract: Thermal-aware optimization of multi-die 3D integrated circuits evaluates many designs, each a costly heat-equation solve. Operator-learning surrogates r

Detecting Money Laundering in Rwandan Mobile Money: A Machine Learning Framework

arXiv:2608.15447v1 Announce Type: new Abstract: Mobile money has widened financial access across Sub-Saharan Africa and enlarged the surface for money-laundering and terrorism-financing (ML/TF) activi

Distinguishing AI-Generated Music from Edited Audio as a Hard-Negative Robustness Task

arXiv:2608.14916v1 Announce Type: cross Abstract: AI-generated music detectors are commonly evaluated against original songs, but real-world uploads are often remixed, re-encoded, pitch-shifted, or ot

applicationsarxiv-cs-ai

Do Uncertainty Signals Help? A Systematic Study of Uncertainty-Aware Decoding with Rollback Mechanisms

arXiv:2608.14653v1 Announce Type: cross Abstract: Prediction uncertainty is a widely adopted metric for quantifying model confidence, with downstream applications spanning model explanation, data sele

researcharxiv-cs-ai

DuplexGen: Decoupling Content, Timing, and Acoustics for Synthetic Dialogue Speech

arXiv:2608.16053v1 Announce Type: new Abstract: Synthetic conversational speech has become an important resource for developing and evaluating conversational speech systems. However, existing dialogue

researcharxiv-cs-cl

Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inference

model-releasesarxiv-cs-cv

arXiv:2608.15282v1 Announce Type: cross Abstract: Earth observation foundation models (EOFMs) are emerging as reusable representation frameworks for data-driven retrieval, prediction and process model

FirstDiff: One-Step Diffusion-Based Anomaly Detection for Multivariate Time Series via Initial Noise Prediction

arXiv:2608.15727v1 Announce Type: cross Abstract: Diffusion models have recently shown strong potential for multivariate time-series anomaly detection by learning the distribution of normal data throu

Free-Flow Class-Incremental Learning: Towards Robust CIL under Variable Class Arrivals

arXiv:2604.02765v2 Announce Type: replace Abstract: Class-incremental learning (CIL) is commonly evaluated under predefined schedules with fixed or nearly equal class increments, leaving irregular cla

safetyarxiv-cs-lg

Grok 4.6 takes top spot on this benchmark

model-releaseselon-musk--x

Grok 4.6 takes top spot on this benchmark 🥇 We evaluated Grok 4.6 on MedAgentBench, a benchmark for agentic clinical EHR tasks, and it took the top spot. Grok 4.6 posts the highest pass@1 we've measur

HarmProfile: Characterizing Harmful Distributions in Frontier LLMs

arXiv:2608.14577v1 Announce Type: cross Abstract: Frontier large language models (LLMs) safety evaluation has largely treated harmful generation as an attack outcome rather than as an object of analys

Historical Backtesting for Scientific Question Discovery: A Protocol and Astronomy Pilot

arXiv:2608.16795v1 Announce Type: cross Abstract: Systems that generate scientific research questions are evaluated today by expert scores, LLM-as-judge ratings, or curated case studies -- all subject

Honeyquest for LLMs: Rethinking Cyber Deception for AI Attackers

model-releasesarxiv-cs-cl

arXiv:2606.21037v2 Announce Type: replace-cross Abstract: The empirical foundation of cyber deception relies on human-centered hypotheses, but the rapid emergence of autonomous, AI-enabled attackers c

Image Denoising via the Adaptive Rank-Cluster Filter

arXiv:2608.15298v1 Announce Type: new Abstract: A spatial-local image-denoising filter is proposed, and its performance metrics are evaluated in comparison with baseline filtering algorithms, includin

researcharxiv-cs-cv

Kozuchi Agent: A Language-Agnostic Open-Weight Agent for Software Repair

arXiv:2608.15579v1 Announce Type: cross Abstract: Industrial software-engineering teams increasingly need LLM agents that turn bug reports into correct patches, yet benchmark-scale operation adds long

LongDocBench: Benchmarking TOC Hierarchy and Contextual Relationship Recovery in Long Documents

arXiv:2608.15064v1 Announce Type: new Abstract: Parsing visual documents into machine-readable representations is fundamental to document intelligence. Existing benchmarks focus on page-level element

MedMCP-Calc: Benchmarking LLMs for Realistic Medical Calculator Scenarios via MCP Integration

arXiv:2601.23049v2 Announce Type: replace Abstract: Medical calculators are fundamental to quantitative, evidence-based clinical practice. However, their real-world use is an adaptive, multi-stage pro

Local Ai

On the Adversarial Robustness of Remote Sensing Semantic Change Detection

arXiv:2608.15267v1 Announce Type: new Abstract: Semantic change detection (SCD) is a bitemporal dense-prediction task that jointly identifies changed regions and their semantic states before and after

local-aiarxiv-cs-cv

Plausible but Not Valid: A Psychometric Audit of LLMs as Synthetic Survey Respondents

arXiv:2608.14606v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as synthetic survey respondents, but existing evaluations ask whether answers look plausible at the

applicationsarxiv-cs-ai

SCOPE: Score-Isolated Agentic Optimization for Video World Models

arXiv:2608.15043v1 Announce Type: new Abstract: Video world models are increasingly used as simulators for planning and embodied decision making, yet improving them at inference time introduces a subt

Second-Order Policy Effects as State Transitions: A Source-Linked Benchmark for Policy Simulation

arXiv:2608.15101v1 Announce Type: new Abstract: Policy evaluation often estimates direct benefits and costs while treating the institutional environment as fixed. In practice, a policy changes the sys

Self-Supervised Topologically Invariant Manifold Learning for Railway Image Quality Assessment

arXiv:2608.15217v1 Announce Type: new Abstract: Existing blind image quality assessment (BIQA) methods typically rely on synthetic distortions and subjective annotations, limiting generalization in re

applicationsarxiv-cs-cv

Skill2Query: Exploiting Skill Structure to Generate Pseudo-Queries for Agent Skill Retrieval

model-releasesarxiv-cs-cl

arXiv:2608.16071v1 Announce Type: new Abstract: Pseudo-query generation can alleviate the supervision bottleneck for agent skill retrieval, but existing document-level approaches typically leave the r

Local Ai

SpIn-ViT: Designing a Sparsity-Induced Vision Transformer That Is Mechanistically Interpretable

arXiv:2608.14922v1 Announce Type: cross Abstract: Mechanistic interpretability has recently expanded to Vision Transformers (ViTs), with Sparse Autoencoders (SAEs) increasingly used as post-hoc tools

local-aiarxiv-cs-ai

Agents

SysEvolve: An AI-native, safe, autonomous adversarial attack-defense co-evolutionary system

arXiv:2608.15012v1 Announce Type: cross Abstract: The rapid advancement of large language models (LLMs) has created a growing asymmetry in cybersecurity, where attack accelerates toward autonomous exe

agentsarxiv-cs-ai

Take it Personally: The Limits of General SSL Representations for Real-Life PPG Emotion Detection

arXiv:2608.14675v1 Announce Type: cross Abstract: While Self-Supervised Learning (SSL) effectively extracts general representations from noisy, unconstrained physiological signals such as photoplethys

applicationsarxiv-cs-ai

TaoLive Digital Avatar Agent Technical Report: Training Agents to Evolve with Their Harness

arXiv:2608.15763v1 Announce Type: new Abstract: AI-powered digital-avatar streamers in live e-commerce must answer product questions, engage viewers, and execute changing business strategies in real t

safetyarxiv-cs-cl

The Fragility of Strategic Thinking in Large Language Models

arXiv:2510.10813v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly applied to domains that require reasoning about other agents' behavior, such as negotiation, policy de

The Quality of Claude AI-authored Python Tests Is Not Weaker Than Human-authored Tests

arXiv:2608.15188v1 Announce Type: cross Abstract: We evaluate the quality of Claude AI-written Python tests against human-written Python tests from two established open-source projects Django and Pand

Time-Aware Validation of Machine Learning Fuel Consumption Models: Evidence from 1,Hz Operational Data, CCGS extit{Sir Wilfrid Laurier}

arXiv:2608.16833v1 Announce Type: new Abstract: Ship fuel consumption (SFC) prediction supports vessel operation optimisation, emissions estimation, and decision support systems (DSS) for sustainable

applicationsarxiv-cs-lg

Toward Better Assessment of LLMs' Performance in Clinical Error Detection

arXiv:2608.16643v1 Announce Type: cross Abstract: Automated detection of errors in clinical documentation is a promising application of large language models (LLMs), yet decisions to deploy such model

Local Ai

TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation

arXiv:2608.16765v1 Announce Type: cross Abstract: Despite recent advances in unified multimodal models for multi-reference image generation, existing benchmarks remain organized around predefined task

local-aiarxiv-cs-ai

Trajectory-Level Automatic Curriculum Learning for Legged Locomotion on Unstructured Terrain

arXiv:2608.16164v1 Announce Type: new Abstract: Training locomotion policies for complex unstructured terrain requires a curriculum to avoid early exploration failures. However, since unstructured ter

TwinGridShield: Consequence-Aware Runtime Authorization for LLM Grid-Agent Actions

arXiv:2608.15391v1 Announce Type: new Abstract: Large language model (LLM)-assisted energy-management tools can translate natural-language context into structured grid commands, but syntactic validity

VARM-Bench: Benchmarking Verifiable Structured Reasoning in Chinese Abusive Speech Moderation

arXiv:2608.15600v1 Announce Type: new Abstract: The widespread circulation of abusive online content has increased the need for reliable moderation of Chinese social-media text. Existing Chinese bench

VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models

arXiv:2603.24575v2 Announce Type: replace-cross Abstract: Scalable Vector Graphics (SVG) are essential for technical illustration and digital design, offering resolution independence and semantic edit

WANDR: A Benchmark for Wide and Deep Research

arXiv:2608.14747v1 Announce Type: new Abstract: WANDR (Wide ANd Deep Research) is a benchmark of 500 realistic, challenging data-collection tasks for research agents. Each task requires a system to di

When Single-Dataset Conclusions Fail: A 45-Task Study of Threshold Tuning and Resampling for Imbalanced Classification

arXiv:2608.16147v1 Announce Type: new Abstract: Class-imbalance handling is routinely evaluated on a single benchmark dataset, and the resulting conclusions are reported as if they were properties of