Search: “evaluation”

Iterative tensor network transformations for element-wise evaluation of elementary and filtering functions

arXiv:2608.17135v1 Announce Type: cross Abstract: Tensor networks are powerful formats for compressing large-scale data. However, their application to general data processing has been limited by the d

19 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challenges

DGX agent

arXiv:2608.17605v1 Announce Type: cross Abstract: Conversational AI is moving beyond isolated text prompts toward sustained, multimodal interaction. In real conversations, users clarify goals, revise

safetyarxiv-cs-ai

19 Aug 2026

Benchmarking Quantum Machine Learning for Power-System Attack Detection: Evaluation Choices Decide the Outcome Before the Models Do

model-releasesarxiv-cs-lg

arXiv:2608.15617v1 Announce Type: new Abstract: Machine-learning detectors for power-system cyberattacks are themselves attack surfaces, and quantum machine learning has been proposed for them. We ben

Inferential Evaluation of Surrogate-Derived Models under Covariate Shift

arXiv:2608.15783v1 Announce Type: cross Abstract: In transfer-learning settings, a model derived from abundant surrogate labels may be deployed in a target population where gold-standard outcomes are

safetyarxiv-cs-lg

Security of Foundation-Model-Powered Embodied Agents: Attack Surfaces, Attacks, Defenses, and Evaluation

arXiv:2608.16843v1 Announce Type: new Abstract: Foundation models are increasingly used for perception, reasoning, planning, and action generation in embodied agents, creating security risks that can

agentsarxiv-cs-ro

Towards Standardized Evaluation in Automated Domain Modeling: Introducing a Benchmark

arXiv:2608.15255v1 Announce Type: new Abstract: Domain modeling plays an essential role in domain-driven design, capturing essential entities and their relationships within a specific domain. Despite

Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Dissemination

arXiv:2608.14391v1 Announce Type: cross Abstract: Recent video generators can fabricate realistic depictions of wars, disasters, public emergencies, and other real-world crises, creating substantial r

17 Aug 2026

Hardware

Enhancing Virtual Agents through SLMs and Edge-Computing: An Exploratory Evaluation of Think and Memory Processes

arXiv:2608.13420v1 Announce Type: new Abstract: Embodied intelligent virtual agents are expected to operate as persistent, adaptive, and context-aware entities within complex virtual and Metaverse wor

hardwarearxiv-cs-ai

14 Aug 2026

Group Alignment-Induced Sycophancy: A Two-Sided Evaluation of Steerable Pluralistic Alignment

arXiv:2608.11528v1 Announce Type: new Abstract: Group alignment adapts a language model to a demographic group to produce responses that reflect the group's opinions, values, and preferences. Sycophan

safetyarxiv-cs-cl

13 Aug 2026

Evaluation-Conditioned Training: Teaching Models to Generalize to Stronger Oversight Regimes

arXiv:2608.10209v1 Announce Type: new Abstract: Feedback signals used to train Large Language Models (LLMs) are the primary driver of their behavior and our main lever for instilling alignment with hu

safetyarxiv-cs-ai

12 Aug 2026

Seeds Before Objectives: Rethinking Evaluation for Low-Resource Garhwali ASR

arXiv:2608.10670v1 Announce Type: new Abstract: At corpus sizes typical of low-resource dialects, single-run comparisons can yield gains that do not replicate. We show this for Garhwali, an under-reso

12 Aug 2026

Do Evaluation Metrics Detect Errors in Classical Chinese to English Translations?

arXiv:2608.08283v1 Announce Type: cross Abstract: Although large language models can translate some historical languages surprisingly well, their usefulness in digital humanities workflows is limited

ELICITED: EHR-grounded Longitudinal Interactive Conversations for Information-seeking Triage Evaluation and Decision-making

arXiv:2608.09024v1 Announce Type: new Abstract: Emergency-department (ED) triage requires clinicians to rapidly identify patients who need immediate attention, determine who can safely wait, and prior

Illusion or Integrity? Geometrical Consistency Metric for AIGC Video Quality Evaluation

arXiv:2608.09594v1 Announce Type: cross Abstract: Recently, AI-driven video generation has attracted considerable attention. This surge increases the demand for reliable video quality assessment (VQA)

Right Answer, Wrong Heat: Explanation-Aware Evaluation and Thermal-Grounded Feedback for MLLMs on Infrared Images

arXiv:2608.09145v1 Announce Type: new Abstract: General-purpose multimodal large language models (MLLMs) are increasingly applied to infrared images, where they are commonly scored by answer accuracy

The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong World

arXiv:2608.08239v1 Announce Type: cross Abstract: LLM routers promise efficiency by matching each request to the cheapest adequate model, and are increasingly applied per step inside multi-step agents

agentsarxiv-cs-cl

Who Verifies the Benchmark? Decentralizing Trust in Large Language Model Evaluation

arXiv:2608.07762v1 Announce Type: new Abstract: LLM benchmarks can build an organization's reputation and attract customers, but only when results are transparent and verifiable. Unverified claims tha

An Exploratory Evaluation of LLM-Assisted Rewriting of Moderate-Complexity Financial Sentences for DisCoCat-Based Sentiment Analysis

arXiv:2608.07439v1 Announce Type: new Abstract: Quantum natural language processing (QNLP) provides a grammar-aware framework for text modeling, and Distributional Compositional Categorical (DisCoCat)

10 Aug 2026

IceHorizon: A Dataset for Horizon Detection in Ice-Covered Maritime Environments and Comparative Evaluation of Detection Methods

arXiv:2608.07018v1 Announce Type: cross Abstract: Horizon detection in images of ice-covered waters is a challenging problem for maritime navigation due to low contrast between water and sky, cluttere

10 Aug 2026

The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agents

arXiv:2608.06663v1 Announce Type: new Abstract: Frontier language models solve reasoning problems in a single forward pass that would have been research contributions years ago, yet fail at multi-hour

safetyarxiv-cs-cl

10 Aug 2026

A Unified Causal Inference Framework for the Desirability of Outcome Ranking Paradigm in Benefit-Risk Evaluation

arXiv:2608.05244v1 Announce Type: cross Abstract: We developed a unified covariate-adjusted causal inference framework for estimating the desirability of outcome ranking (DOOR) probability for benefit

safetyarxiv-cs-lg

A Unified Risk View of Uncertainty: Posterior Risk for Disentanglement and Evaluation Beyond Proxies

model-releasesarxiv-cs-lg

arXiv:2608.05995v1 Announce Type: new Abstract: Reliable uncertainty estimates are critical in safety-sensitive applications, where understanding the sources of predictive uncertainty is essential. Th

ASTELD: A Six-Axis Classification Framework for Autonomous AI Agents - Design, Evaluation, and an OpenClaw Case Study

arXiv:2608.05201v1 Announce Type: cross Abstract: Autonomous AI agent platforms differ substantially in architecture, security, tool integration, execution, autonomy, and deployment, yet the field lac

agentsarxiv-cs-ai

Design and Evaluation of a Touchscreen-Based Teleoperation Interface for Robotic Manipulators

arXiv:2608.06219v1 Announce Type: new Abstract: Intuitive teleoperation interfaces are crucial for the safe and effective operation of robotic manipulators in challenging environments. In the nuclear

agentsarxiv-cs-ro

Does Latent Context Help? A Controlled Evaluation of Inverse Reinforcement Learning in Arctic Shipping

arXiv:2608.06105v1 Announce Type: cross Abstract: Artificial Intelligence (AI)-assisted navigation can help Arctic shipping adapt to rapidly changing sea-ice conditions, but reliable deployment requir

safetyarxiv-cs-ai

Evaluation Pitfalls and Sparsity Limitations in LLM-based Confidence Estimates for Classification

arXiv:2608.04899v1 Announce Type: new Abstract: Confidence estimation is essential when LLMs are used for classification, indicating when predictions can be trusted. However, common approaches such as

researcharxiv-cs-cl

6 Aug 2026

Revealed Rationality: Label-Free Evaluation and Regularization from Representation Theorems

arXiv:2608.05015v1 Announce Type: cross Abstract: Representation theorems in decision theory establish that behavior satisfies certain axioms if and only if it can be rationalized by a well-defined ob

6 Aug 2026

LDU-Bench: Multimodal LLM Evaluation for Lithography Defect Understanding under Layout-Varying Circuit Backgrounds

model-releasesarxiv-cs-cv

arXiv:2608.03078v1 Announce Type: new Abstract: Multimodal large language models have demonstrated strong defect recognition capability in industrial anomaly detection. However, in lithography review,

5 Aug 2026

Shorter Reasoning, Earlier Answers? An Evaluation of Reasoning Interfaces

arXiv:2608.03401v1 Announce Type: cross Abstract: Large language models often reason at length before answering, increasing cost and latency. Prompts and trained settings can shorten this reasoning, b

5 Aug 2026

WorldCup Arena: Prospective, Leakage-Free Evaluation of Frontier LLMs on a Live Tournament

arXiv:2608.04008v1 Announce Type: new Abstract: Benchmarks that measure the forecasting ability of large language models are almost always retrospective: the event has happened, the answer is somewher

5 Aug 2026

Local Ai

Object Detection for Autonomous Driving in Chinese Rural Scenes: An Experimental Study on Real-Synthetic Data Mixing and Model Evaluation

arXiv:2607.27058v1 Announce Type: new Abstract: Currently, autonomous driving object detection models face significant data scarcity and generalization challenges when navigating complex Chinese rural

local-aiarxiv-cs-cv

30 Jul 2026

Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response

arXiv:2607.25379v1 Announce Type: new Abstract: Cyber-capable AI agents combine language models with tools, memory, and execution en- vironments to perform multi-step offensive-security tasks. Existin

agentsarxiv-cs-ai

29 Jul 2026

Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models

arXiv:2607.25907v1 Announce Type: cross Abstract: Activation steering controls model behavior by editing internal activations at inference time. We study its input-side dual: optimizing a fluent promp

29 Jul 2026

Vector-Valued Distributional Reinforcement Learning Policy Evaluation: A Hilbert Space Embedding Approach

arXiv:2601.18952v2 Announce Type: replace Abstract: We propose an (offline) multi-dimensional distributional reinforcement learning framework (KE-DRL) that leverages Hilbert space mappings to estimate

safetyarxiv-cs-lg

29 Jul 2026

Local Ai

Beyond Local Inspection: Global, Guideline-Grounded Evaluation of Post-hoc XAI Methods for ECG Classification

arXiv:2607.24035v1 Announce Type: cross Abstract: Explainable AI (XAI) is used to assess whether artificial intelligence models rely on meaningful patterns, yet explanations that appear plausible for

local-aiarxiv-cs-lg

Evaluation of Blood Vessel Segmentation Methods on Hard-to-Detect Vascular Structures

model-releasesarxiv-cs-cv

arXiv:2406.13128v2 Announce Type: replace Abstract: Due to the intricate structure of vascular trees, minor segmentation errors can significantly alter connectivity patterns and increase variability i

ParBench: A Benchmark for Reliable Evaluation of LLM Parallel Code Translation

arXiv:2607.22588v1 Announce Type: new Abstract: Modern compute-intensive software must migrate across a changing ecosystem of accelerators, programming APIs, compiler stacks, and portability layers, i

The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluation

arXiv:2607.22585v1 Announce Type: new Abstract: Public leaderboards for coding agents typically rank systems by model name and pass rate, while the surrounding harness (the scaffold that issues tools,

Trustworthy Medical Segmentation: Uncertainty-Aware U-Net Evaluation Under Clinical Image Degradation

model-releasesarxiv-cs-cv

arXiv:2607.22727v1 Announce Type: new Abstract: Medical image segmentation models often report high benchmark accuracy under ideal imaging conditions, yet their failures under clinical degradation can

Agentic Evaluation of Copyright Law Compliance

arXiv:2607.21799v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly perform commercial tasks that involve retrieving external content such as images and, where appropriate,

27 Jul 2026

Applications

Teachy Mini: Development and Preliminary Evaluation of a Knowledge-Based Generative Social Robot for Higher Education

arXiv:2607.22345v1 Announce Type: new Abstract: Generative social robots (GSRs) powered by large language models offer new possibilities for personalized tutoring in higher education, but also introdu

applicationsarxiv-cs-ro

27 Jul 2026

A Comparative Evaluation of Embeddings and LLMs in a Greek Book Publisher Setting - The CUP Dataset

arXiv:2607.21274v1 Announce Type: cross Abstract: We present CUP, a Greek book retrieval benchmark consisting of 868 catalog records and 104 expert-annotated queries with graded relevance judgments. W

24 Jul 2026

Evaluation and Prognostic Validation of Deep Regression Models for WSI-Based Gene-Expression Prediction

arXiv:2410.00945v2 Announce Type: replace-cross Abstract: Gene-expression profiling is widely used in research and central to many areas of precision oncology, but remains costly and not universally a

24 Jul 2026

Verifier-First Evaluation of Agentic LLMs for Infrastructure-as-Code Generation

arXiv:2607.20478v1 Announce Type: cross Abstract: Infrastructure-as-Code (IaC) generation from natural language requires satisfying provider schemas, dependency planning, and organizational policy con

24 Jul 2026

A Multi-Dimensional Evaluation of Explainability in Media Bias Detection

arXiv:2607.19954v1 Announce Type: new Abstract: Detecting media bias automatically is difficult because biased framing is often subtle, yet in domains such as news analysis, accurate predictions alone

safetyarxiv-cs-cl

Attention Without Grounding: Causal Evaluation of Visual Explanations in Medical VLMs

arXiv:2607.18577v1 Announce Type: new Abstract: Attention and saliency heatmaps are widely used to explain medical Vision-Language Model (VLM) outputs on chest X-rays, yet whether they truly highlight

Crashing Waves vs. Rising Tides: Findings on AI Automation from Thousands of Worker Evaluations of Labor Market Tasks

arXiv:2604.01363v2 Announce Type: replace Abstract: We propose that AI automation is a continuum between: (i) crashing waves where AI capabilities surge abruptly over small sets of tasks, and (ii) ris

FORCE-Bench: A Benchmark, Dataset, and Evaluation Harness for Agentic AI in Enterprise Finance

arXiv:2607.19409v1 Announce Type: new Abstract: Recent advances in large language models have accelerated deployment of agentic systems in operational finance. Existing benchmarks emphasize measuring