Search: “evaluation”

Grading Needs a Rubric, Not Intelligence

arXiv:2608.17938v1 Announce Type: cross Abstract: Small language models can grade open-ended examination answers as reliably as substantially more expensive models when they grade against an explicit

MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2608.17659v1 Announce Type: cross Abstract: LLM-powered GUI agents that autonomously operate smartphones are rapidly transitioning from research prototypes to early real-world deployment. Howeve

model-releasesarxiv-cs-ai

19 Aug 2026

PathoArgus: Advancing Evidence-Grounded Long-Context Visual Reasoning across Gigapixel Whole-Slide and Multi-Slide Case Contexts

model-releasesarxiv-cs-cv

arXiv:2608.17607v1 Announce Type: new Abstract: Whole-slide pathology reasoning requires models to integrate gigapixel-scale visual evidence across complete case-linked slides, yet current question-an

Revisiting WEASEL 2.0: Reproduction, Sensitivity, and an Adaptive Ensemble-Size Rule

arXiv:2608.18021v1 Announce Type: new Abstract: WEASEL 2.0 is a dictionary-based time series classifier that combines dilated sliding windows with a randomised hyperparameter ensemble and a fixed-size

researcharxiv-cs-lg

Risk-Controllable Multi-View Diffusion for Driving Scenario Generation

arXiv:2603.11534v2 Announce Type: replace Abstract: Generating safety-critical driving scenarios is crucial for evaluating and improving autonomous driving systems, but long-tail risky situations are

safetyarxiv-cs-cv

Runtime Governance for Agentic AI: Action-Boundary Control with Trusted Provenance and Fail-Closed Execution

arXiv:2608.16891v1 Announce Type: new Abstract: Agentic AI systems request tool actions that can modify files, send messages, launch jobs, or change workflow state. This shifts the safety problem from

safetyarxiv-cs-ai

Agents

SkillEffect: Checked Lowering for Memory-Bounded Agent Tools

arXiv:2608.17007v1 Announce Type: new Abstract: Agent Skills can specify procedural and resource obligations for tool use, and language models instantiate them as concrete programs. However, when mode

agentsarxiv-cs-ai

SPSA Hyperparameter Tuning for Variational Quantum Natural Language Inference

arXiv:2608.16939v1 Announce Type: cross Abstract: Training variational quantum models requires choosing between parameter-shift gradients, which are exact but cost O(P) forward evaluations, and simult

Toward Personal Intelligence Through Cooperative Observation

arXiv:2608.17128v1 Announce Type: new Abstract: A personal AI system needs a model of the user's goals, constraints, and ongoing commitments to plan and act on their behalf, and the quality of that mo

Toward the Optimal Regret-Instability Trade-off in Multi-Armed Bandits

arXiv:2608.17841v1 Announce Type: cross Abstract: Multi-armed bandit algorithms are evaluated by regret, yet comparable regret can coexist with different allocations across independent runs. We study

researcharxiv-cs-lg

TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation

arXiv:2608.17588v1 Announce Type: new Abstract: Agent Skills package reusable natural language procedures with executable resources, enabling software agents to acquire task specific capabilities with

Wuying-Browser-Agent: Real-World Centric Fundamental Long-Horizon Browser Agents

arXiv:2608.17319v1 Announce Type: new Abstract: Browser agents perform well on short, clean demonstrations, but real deployment is fundamentally different: agents must sustain dozens of decisions on l

A Human-LLM Teaming Framework for Privacy Risk Analysis: An Illustration with CBDC-Based Welfare Schemes

arXiv:2608.16461v1 Announce Type: cross Abstract: Central Bank Digital Currency (CBDC)-based welfare schemes may be potentially privacy invasive as they process significant volumes of beneficiary pers

AnchorScore: A CLIP-Based Diagnostic of MLLM Annotation Difficulty

arXiv:2608.16690v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are widely used for automated annotation, yet their per-class accuracy varies widely (e.g., 12%-98% across the

researcharxiv-cs-cv

Automatic or Controlled? Repetition Priming Reveals Divergent Processing in Base LLMs, Instruct LLMs, and Humans

arXiv:2608.14681v1 Announce Type: cross Abstract: Words recur constantly in natural language use, yet it remains unclear whether language models reactivate prior representations or re-evaluate repeate

Beat the Counter First: A Baseline for Temporal-Graph Anomaly Detectors

arXiv:2608.15965v1 Announce Type: new Abstract: Progress in streaming, edge-level graph anomaly detection (GAD) has been marked by increasingly elaborate architectures, from count-min-sketch chi squar

Benchmarking Frontier Text-to-Image Models on Image-Description Prompts

model-releasesarxiv-cs-cv

arXiv:2608.14976v1 Announce Type: new Abstract: Text-to-image models are typically reported on average-case prompts, which understates the gap between systems on compositionally demanding requests inv

CAPO: Constraint-Aware Prompt Optimization for LLM Agents

arXiv:2608.16068v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed as agents that rely on system prompts to use tools and complete tasks. Such deployments impose

safetyarxiv-cs-ai

Agents

Dear Algo: A Precision-First Agentic Intent Layer for Unified Search and Recommendation

arXiv:2608.15877v1 Announce Type: new Abstract: Search and recommendation serve a shared discovery objective but encode intent differently. We study this boundary through Dear Algo on Threads, a deplo

agentsarxiv-cs-ai

DeepOHeat-v2: Self-Improving Operator Learning for Fast and Trustworthy Thermal Optimization in 3D-IC Design

arXiv:2608.16080v1 Announce Type: new Abstract: Thermal-aware optimization of multi-die 3D integrated circuits evaluates many designs, each a costly heat-equation solve. Operator-learning surrogates r

Detecting Money Laundering in Rwandan Mobile Money: A Machine Learning Framework

arXiv:2608.15447v1 Announce Type: new Abstract: Mobile money has widened financial access across Sub-Saharan Africa and enlarged the surface for money-laundering and terrorism-financing (ML/TF) activi

Distinguishing AI-Generated Music from Edited Audio as a Hard-Negative Robustness Task

arXiv:2608.14916v1 Announce Type: cross Abstract: AI-generated music detectors are commonly evaluated against original songs, but real-world uploads are often remixed, re-encoded, pitch-shifted, or ot

applicationsarxiv-cs-ai

Do Uncertainty Signals Help? A Systematic Study of Uncertainty-Aware Decoding with Rollback Mechanisms

arXiv:2608.14653v1 Announce Type: cross Abstract: Prediction uncertainty is a widely adopted metric for quantifying model confidence, with downstream applications spanning model explanation, data sele

DuplexGen: Decoupling Content, Timing, and Acoustics for Synthetic Dialogue Speech

arXiv:2608.16053v1 Announce Type: new Abstract: Synthetic conversational speech has become an important resource for developing and evaluating conversational speech systems. However, existing dialogue

researcharxiv-cs-cl

Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inference

model-releasesarxiv-cs-cv

arXiv:2608.15282v1 Announce Type: cross Abstract: Earth observation foundation models (EOFMs) are emerging as reusable representation frameworks for data-driven retrieval, prediction and process model

FirstDiff: One-Step Diffusion-Based Anomaly Detection for Multivariate Time Series via Initial Noise Prediction

arXiv:2608.15727v1 Announce Type: cross Abstract: Diffusion models have recently shown strong potential for multivariate time-series anomaly detection by learning the distribution of normal data throu

Free-Flow Class-Incremental Learning: Towards Robust CIL under Variable Class Arrivals

arXiv:2604.02765v2 Announce Type: replace Abstract: Class-incremental learning (CIL) is commonly evaluated under predefined schedules with fixed or nearly equal class increments, leaving irregular cla

safetyarxiv-cs-lg

HarmProfile: Characterizing Harmful Distributions in Frontier LLMs

arXiv:2608.14577v1 Announce Type: cross Abstract: Frontier large language models (LLMs) safety evaluation has largely treated harmful generation as an attack outcome rather than as an object of analys

Historical Backtesting for Scientific Question Discovery: A Protocol and Astronomy Pilot

arXiv:2608.16795v1 Announce Type: cross Abstract: Systems that generate scientific research questions are evaluated today by expert scores, LLM-as-judge ratings, or curated case studies -- all subject

Honeyquest for LLMs: Rethinking Cyber Deception for AI Attackers

model-releasesarxiv-cs-cl

arXiv:2606.21037v2 Announce Type: replace-cross Abstract: The empirical foundation of cyber deception relies on human-centered hypotheses, but the rapid emergence of autonomous, AI-enabled attackers c

Image Denoising via the Adaptive Rank-Cluster Filter

arXiv:2608.15298v1 Announce Type: new Abstract: A spatial-local image-denoising filter is proposed, and its performance metrics are evaluated in comparison with baseline filtering algorithms, includin

researcharxiv-cs-cv

Kozuchi Agent: A Language-Agnostic Open-Weight Agent for Software Repair

arXiv:2608.15579v1 Announce Type: cross Abstract: Industrial software-engineering teams increasingly need LLM agents that turn bug reports into correct patches, yet benchmark-scale operation adds long

LongDocBench: Benchmarking TOC Hierarchy and Contextual Relationship Recovery in Long Documents

arXiv:2608.15064v1 Announce Type: new Abstract: Parsing visual documents into machine-readable representations is fundamental to document intelligence. Existing benchmarks focus on page-level element

MedMCP-Calc: Benchmarking LLMs for Realistic Medical Calculator Scenarios via MCP Integration

arXiv:2601.23049v2 Announce Type: replace Abstract: Medical calculators are fundamental to quantitative, evidence-based clinical practice. However, their real-world use is an adaptive, multi-stage pro

Local Ai

On the Adversarial Robustness of Remote Sensing Semantic Change Detection

arXiv:2608.15267v1 Announce Type: new Abstract: Semantic change detection (SCD) is a bitemporal dense-prediction task that jointly identifies changed regions and their semantic states before and after

local-aiarxiv-cs-cv

Plausible but Not Valid: A Psychometric Audit of LLMs as Synthetic Survey Respondents

arXiv:2608.14606v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as synthetic survey respondents, but existing evaluations ask whether answers look plausible at the

applicationsarxiv-cs-ai

SCOPE: Score-Isolated Agentic Optimization for Video World Models

arXiv:2608.15043v1 Announce Type: new Abstract: Video world models are increasingly used as simulators for planning and embodied decision making, yet improving them at inference time introduces a subt

Second-Order Policy Effects as State Transitions: A Source-Linked Benchmark for Policy Simulation

arXiv:2608.15101v1 Announce Type: new Abstract: Policy evaluation often estimates direct benefits and costs while treating the institutional environment as fixed. In practice, a policy changes the sys

Self-Supervised Topologically Invariant Manifold Learning for Railway Image Quality Assessment

arXiv:2608.15217v1 Announce Type: new Abstract: Existing blind image quality assessment (BIQA) methods typically rely on synthetic distortions and subjective annotations, limiting generalization in re

applicationsarxiv-cs-cv

Skill2Query: Exploiting Skill Structure to Generate Pseudo-Queries for Agent Skill Retrieval

model-releasesarxiv-cs-cl

arXiv:2608.16071v1 Announce Type: new Abstract: Pseudo-query generation can alleviate the supervision bottleneck for agent skill retrieval, but existing document-level approaches typically leave the r

Local Ai

SpIn-ViT: Designing a Sparsity-Induced Vision Transformer That Is Mechanistically Interpretable

arXiv:2608.14922v1 Announce Type: cross Abstract: Mechanistic interpretability has recently expanded to Vision Transformers (ViTs), with Sparse Autoencoders (SAEs) increasingly used as post-hoc tools

local-aiarxiv-cs-ai

Agents

SysEvolve: An AI-native, safe, autonomous adversarial attack-defense co-evolutionary system

arXiv:2608.15012v1 Announce Type: cross Abstract: The rapid advancement of large language models (LLMs) has created a growing asymmetry in cybersecurity, where attack accelerates toward autonomous exe

agentsarxiv-cs-ai

Take it Personally: The Limits of General SSL Representations for Real-Life PPG Emotion Detection

arXiv:2608.14675v1 Announce Type: cross Abstract: While Self-Supervised Learning (SSL) effectively extracts general representations from noisy, unconstrained physiological signals such as photoplethys

applicationsarxiv-cs-ai

TaoLive Digital Avatar Agent Technical Report: Training Agents to Evolve with Their Harness

arXiv:2608.15763v1 Announce Type: new Abstract: AI-powered digital-avatar streamers in live e-commerce must answer product questions, engage viewers, and execute changing business strategies in real t

safetyarxiv-cs-cl

The Fragility of Strategic Thinking in Large Language Models

arXiv:2510.10813v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly applied to domains that require reasoning about other agents' behavior, such as negotiation, policy de

safetyarxiv-cs-ai

The Quality of Claude AI-authored Python Tests Is Not Weaker Than Human-authored Tests

arXiv:2608.15188v1 Announce Type: cross Abstract: We evaluate the quality of Claude AI-written Python tests against human-written Python tests from two established open-source projects Django and Pand

Time-Aware Validation of Machine Learning Fuel Consumption Models: Evidence from 1,Hz Operational Data, CCGS extit{Sir Wilfrid Laurier}

arXiv:2608.16833v1 Announce Type: new Abstract: Ship fuel consumption (SFC) prediction supports vessel operation optimisation, emissions estimation, and decision support systems (DSS) for sustainable

applicationsarxiv-cs-lg

Toward Better Assessment of LLMs' Performance in Clinical Error Detection

arXiv:2608.16643v1 Announce Type: cross Abstract: Automated detection of errors in clinical documentation is a promising application of large language models (LLMs), yet decisions to deploy such model