Search: “evaluation”

Security-First Evaluation of Text-to-Terraform: Benchmarking LLMs and SLMs for Secure IaC Generation

arXiv:2608.02672v1 Announce Type: cross Abstract: Cloud misconfiguration remains a leading cause of security incidents, yet whether LLMs and SLMs can generate security-compliant Infrastructure-as-Code

5 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

Stochastic Multiple Shooting Trajectory Optimization via Sequential Local Policy Evaluation

DGX agent

arXiv:2608.03978v1 Announce Type: new Abstract: Stochastic single shooting trajectory optimization methods such as Model Predictive Path Integral control (MPPI) have been widely adopted in robotics du

local-aiarxiv-cs-ro

5 Aug 2026

MIEScore: Human-Aligned Evaluation for Multi-Source Image Editing

model-releasesarxiv-cs-cv

arXiv:2608.02059v1 Announce Type: new Abstract: Recent advances in unified multimodal models have significantly improved text-guided image editing abilities. In particular, models such as Nano-Banana-

4 Aug 2026

UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluation

model-releasesarxiv-cs-lg

arXiv:2608.00915v1 Announce Type: new Abstract: Uplift modeling (conditional-average-treatment-effect estimation) drives personalized targeting, yet published uplift benchmarks frequently disagree on

4 Aug 2026

A Generalized-Bayes Perspective on Counterfactual Explanations: Posterior-Based Decision-Making and Evaluation

arXiv:2607.29077v1 Announce Type: new Abstract: Counterfactual explanations (CEs) enhance the interpretability of machine learning models by identifying the smallest change to an input required to obt

3 Aug 2026

ARB: A Matched Authorship-Rewriting Benchmark Dataset for AI-Text Detector Evaluation

arXiv:2607.29539v1 Announce Type: cross Abstract: Standard AI-text detection benchmarks compare human-written text against text generated directly by large language models (LLMs). While prior work has

3 Aug 2026

The Morphological Core of Dungan: A Two-Dialect Finite-State Model and a Multi-Genre Evaluation

arXiv:2607.28766v1 Announce Type: new Abstract: Dungan, a Sinitic language of Central Asia written in a Cyrillic-based script, is described in detail in the grammatical literature, yet the quantitativ

3 Aug 2026

A systematic evaluation of machine learning classifiers for event-by-event background rejection in LAFOV PET scanners

arXiv:2607.25732v1 Announce Type: new Abstract: The introduction of LAFOV PET scanners brings significant sensitivity gains but also a substantial increase in the background rate from accidental coinc

researcharxiv-cs-cv

29 Jul 2026

Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applications

arXiv:2607.25642v1 Announce Type: cross Abstract: Instruction-based Image Editing (IIE) aims to transform a given image into a new one based on textual instructions. Advances in Large Language Models

29 Jul 2026

Bit-Accurate FPGA Evaluation of Learned Feature Gating in a Fixed-Point Fourier-Feature Automatic Modulation Classifier

arXiv:2607.24568v1 Announce Type: new Abstract: Learned feature reweighting can improve automatic modulation classification (AMC) in software, but the same operation introduces additional arithmetic a

researcharxiv-cs-lg

28 Jul 2026

Design and Human Evaluation of Tactile Withdrawal Reflexes for a Skin-Covered Robot Arm

arXiv:2607.22249v1 Announce Type: new Abstract: Nociception is a protective biological mechanism that links harmful stimulation to a reaction. This paper investigates artificial nociception for a robo

safetyarxiv-cs-ro

27 Jul 2026

Phonetic forced alignment for low-resource language varieties: Model training and evaluation on Chengdu Mandarin

arXiv:2607.21332v1 Announce Type: cross Abstract: Phonetic forced alignment is a key technique in phonetic research, yet existing alignment systems lack specialized models for low-resource language va

safetyarxiv-cs-ai

24 Jul 2026

Scale Up Strategically: Learning Compositional Generalization via Bias-Aware Evaluation and Data Collection for Robotic Manipulation

arXiv:2607.21582v1 Announce Type: cross Abstract: Compositional generalization is essential for robot to follow diverse instructions. However, pretrained policies are known to take shortcuts, deferrin

safetyarxiv-cs-cv

24 Jul 2026

A Systematic Evaluation of Traditional Privacy Policy Analysis Tools Against LLMs

arXiv:2607.17075v2 Announce Type: replace-cross Abstract: The advent of LLMs has significantly changed the research on privacy policy and data compliance analysis by enabling tasks that previously req

23 Jul 2026

A Comparative Evaluation of Large Vision-Language Models for 2D Object Detection under SOTIF Conditions

model-releasesarxiv-cs-cv

arXiv:2601.22830v2 Announce Type: replace Abstract: Reliable environmental perception remains one of the main obstacles for safe operation of automated vehicles. Safety of the Intended Functionality (

16 Jul 2026

Can We Steer the Black-Box? Towards Controllability-Centric Evaluation of Recommender Systems with Collaborative Agents

arXiv:2607.13418v1 Announce Type: cross Abstract: Recommender systems operate as Black-Boxes, leaving users and regulators unable to steer their outputs toward specific intentions or audit their behav

16 Jul 2026

Local Ai

Federated Explainable Artificial Intelligence: Roles, Architectures, Evaluation, and Open Challenges

arXiv:2607.13045v1 Announce Type: cross Abstract: Federated Learning (FL) has emerged as a key paradigm for privacy-preserving collaborative model training across distributed and heterogeneous data so

local-aiarxiv-cs-ai

16 Jul 2026

Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents

arXiv:2607.12790v1 Announce Type: new Abstract: Self-evolving agent systems improve by creating, revising, and retiring their own skills, but every such loop rests on a hidden assumption: a reliable e

safetyarxiv-cs-ai

15 Jul 2026

An Online Reference-Free Evaluation Framework for Flowchart Image-to-Code Generation

arXiv:2602.13376v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly used in document processing pipelines to convert flowchart images into structured code (e.g., M

10 Jul 2026

SimRPD: Optimizing Recruitment Proactive Dialogue Agents through Simulator-Based Data Evaluation and Selection

arXiv:2601.02871v3 Announce Type: replace Abstract: Task-oriented proactive dialogue agents play a pivotal role in recruitment, particularly for steering conversations towards specific business outcom

10 Jul 2026

Prior-matched evaluation of operational Earth-observation classifiers: a three-number reporting method demonstrated on Sentinel-1 internal-wave detection

arXiv:2607.07146v1 Announce Type: cross Abstract: The Internal Waves Service screens the Sentinel-1 Wave-mode archive for internal solitary waves, routing detections to experts whose adjudication time

researcharxiv-cs-cv

9 Jul 2026

Rethinking Multimodal Time-Series Forecasting Evaluation

model-releasesarxiv-cs-lg

arXiv:2607.06973v1 Announce Type: new Abstract: We introduce a new context-enriched, multimodal time series forecasting benchmark, TimesX. TimesX contains a wide selection of high-quality real-world t

9 Jul 2026

A Patient Simulation Framework for Risk Assessment of Conversational Healthcare AI: Evaluation of an Antidepressant Decision Aid

arXiv:2602.11391v4 Announce Type: replace Abstract: Objective: This study develops and validates a patient simulation framework that aligns with the National Institute of Standards and Technology (NIS

applicationsarxiv-cs-cl

8 Jul 2026

Data-dependent Evaluations for Budgeted Submodular Maximization

arXiv:2607.05759v1 Announce Type: cross Abstract: Submodular maximization is an important building block for developing algorithms in many areas such as machine learning and data mining. Due to the NP

8 Jul 2026

Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development

arXiv:2607.06074v1 Announce Type: cross Abstract: Prompt engineering has emerged as a critical yet undertaught skill for software developers, one that traditional learning approaches are ill-equipped

8 Jul 2026

FlowEval: Reference-Based Evaluation of Generated User Interfaces

researchapple-ml-research

While large language models (LLMs) and coding agents are often applied to user interface (UI) development, developers find it difficult to reliably assess their proficiency in visual and interaction d

How Do Diffusion Classifiers Decide? A Bias-Centric Evaluation

arXiv:2607.03831v1 Announce Type: cross Abstract: Diffusion models have recently been repurposed for zero-shot classification, giving rise to diffusion classifiers that identify the best-matching text

Information-Geometric Superposed Vowel Evaluation: Part 1. Moraic Syllabary (Japanese)

arXiv:2607.04154v1 Announce Type: cross Abstract: This paper explains the principles and provides examples of a new method for distinguishing between FAKE human speech synthesized by generative AI and

Rating the Pitch, Not the Product: User Evaluations of LLMs Reflect Expectations More Than Performance

arXiv:2607.05113v1 Announce Type: new Abstract: Imagine two users interact with the same LLM. One has been told it is the cutting-edge flagship model; the other, an older, weaker model. They walk away

researcharxiv-cs-cl

U-Joint CAAMS: Experimental Evaluation of a Universal-Joint Continuum Manipulator for Aerial Manipulation

model-releasesarxiv-cs-ro

arXiv:2607.03321v1 Announce Type: new Abstract: Continuum manipulators mounted on multi-rotor UAVs enable compliant aerial manipulation, but payloads and propeller downwash amplify out-of-plane bendin

Assessing VLM Reliability for Medical Image Quality Evaluation Under Corruption and Bias

arXiv:2607.01973v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly applied in medical tasks such as pathology description, report generation, and visual question answerin

3 Jul 2026

CLAP: Closed-Loop Training, Evaluation, and Release Control for Domain Agent Post-training

arXiv:2607.01846v1 Announce Type: new Abstract: Domain agents often face noisy business data, uncertain post-training gains, offline/application mismatch, and adapter-release risk. This paper presents

3 Jul 2026

Predictive Conformal Slip Monitoring: An Empirical Evaluation of Rolling Split Conformal Prediction for Pre-Incident Traction Loss Detection

arXiv:2607.02124v1 Announce Type: new Abstract: Conventional traction control architectures intervene only after the adhesion limit of a tire has already been breached. This paper investigates whether

researcharxiv-cs-lg

3 Jul 2026

KAGE-Bench: Fast Known-Axis Visual Generalization Evaluation for Reinforcement Learning

arXiv:2601.14232v2 Announce Type: replace-cross Abstract: Pixel-based reinforcement learning agents often fail under purely visual distribution shift even when latent dynamics and rewards are unchange

Quantifying the Affective Gap: A Zero-Shot Evaluation of LLMs on Fine-Grained Emotion Taxonomies

arXiv:2607.00968v1 Announce Type: new Abstract: Emotion recognition in natural language is a foundational challenge in affective computing, with critical implications for human-computer interaction, m

SEFORA: Student Essays with Feedback Corpus and LLM Feedback Evaluation Framework

arXiv:2607.00274v1 Announce Type: cross Abstract: Effective writing feedback is among the strongest drivers of student learning, yet producing it at scale is labor-intensive. LLMs offer a natural path

Visualizing Impedance Control in Augmented Reality for Teleoperation: Design and User Evaluation

arXiv:2603.25418v2 Announce Type: replace Abstract: Teleoperation for contact-rich manipulation remains challenging, especially when using low-cost, motion-only interfaces that provide no haptic feedb

researcharxiv-cs-ro

WIDER-FAIR: An Annotated Version of the WIDER-FACE Dataset for Fairness Evaluation

model-releasesarxiv-cs-cv

arXiv:2606.31704v1 Announce Type: new Abstract: The deployment of face detection models in real-world applications raises important fairness concerns, as these systems may showcase performance dispari

1 Jul 2026

CLQT: A Closed-Loop, Cost-Aware, Strategy-Consistent Benchmark for Diagnostic Evaluation of LLM Portfolio-Management Agents

arXiv:2606.29771v1 Announce Type: new Abstract: LLM agents are increasingly cast as autonomous portfolio managers, and benchmarks have moved from financial question-answering to sequential trading. Ye

EMPATH: A Multilingual Auditor-Judge Benchmark for Safety Evaluation of Emotional-Support Chatbots

arXiv:2606.30256v1 Announce Type: new Abstract: Safety benchmarks often buy scalability by fixing the prompt, the language, and the turn structure. For emotional-support chatbots, that bargain hides p

Measuring Graph-to-Graph Semantic Similarity in Knowledge Graphs: An Empirical Evaluation of Knowledge Graph Embeddings

arXiv:2606.29180v1 Announce Type: new Abstract: A Knowledge Graph (KG) represents facts as structured triples and is widely used to organize relational knowledge across diverse domains. Just as textua

Structured Proper Loss Geometries for Multiclass Classification: Theory and Controlled Empirical Evaluation

arXiv:2606.29471v1 Announce Type: new Abstract: Strictly proper scoring rules identify the true conditional class distribution at population level, but their curvature can alter optimization and finit

researcharxiv-cs-lg

Towards Evaluation of Implicit Software World Models in Coding LLMs

arXiv:2606.27406v1 Announce Type: cross Abstract: Software engineering, whether performed by humans or by AI agents, requires reasoning about how software behaves. We call the internal model that supp

29 Jun 2026

Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents

arXiv:2606.26479v1 Announce Type: cross Abstract: Recent work (2024 to 2026) has converged on a strategy for defending tool-using LLM agents against indirect prompt injection: rather than training the

Evaluation-Strategy Gap in Fault Diagnosis of Deep Learning Programs

arXiv:2606.26492v1 Announce Type: cross Abstract: Deep Learning (DL) programs can fail during training for many reasons, and diagnosing the cause is a costly and time-consuming maintenance task. Techn

HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualization

arXiv:2606.26614v1 Announce Type: cross Abstract: Large language model (LLM) agents enable natural language interaction for scientific visualization (SciVis). Still, prior systems have essentially pri

Neural Speaker Diarization via Multilingual Training: Evaluation on Low-Resource Nepali-Hindi Speech

arXiv:2606.26144v1 Announce Type: cross Abstract: Speaker diarization, the task of determining 'who spoke when' in a multi-speaker recording, is a critical component in applications such as meeting tr

safetyarxiv-cs-cl

AutoRelAnnotator: Calibrated Model Cascades for Cost-Efficient Relevance Evaluation in Sponsored Search

arXiv:2606.25871v1 Announce Type: cross Abstract: How can we generate high-quality relevance annotations at scale without the cost and delays of human labeling? Relevance annotations are the backbone