Search: “evaluation”

External Clustering Validation by the Homogeneity-Parsimony Trade-off

arXiv:2607.20799v1 Announce Type: new Abstract: Scalar metrics are often used to evaluate clusterings against known classes, but they can obscure a fundamental trade-off: clusterings should be informa

researcharxiv-cs-lg

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Future Rendering neq Future Surface: A Benchmark and Dataset for Dynamic Surface Reconstruction Beyond the Observed Window

DGX agent

arXiv:2607.21471v1 Announce Type: new Abstract: Dynamic-scene reconstruction is almost always evaluated inside the observed time window, yet deployment settings such as AR overlays, robot interaction,

model-releasesarxiv-cs-cv

24 Jul 2026

GlucoTune: A Unified Framework for Blood Glucose Preprocessing, Forecasting, and Benchmarking in Diabetes

arXiv:2607.21117v1 Announce Type: cross Abstract: Preprocessing blood glucose time-series data is a critical yet often overlooked step in developing data-driven methods for diabetes management, partic

researcharxiv-cs-ai

Agents

GRADRAG: Cross-Component Prompt Adaptation for Coordinated Multi-Agent RAG

arXiv:2607.21324v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems increasingly employ multiple LLM agents. Yet, most prior work optimizes components in isolation rather th

agentsarxiv-cs-ai

InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents

arXiv:2607.20468v1 Announce Type: new Abstract: AI agents are increasingly used to automate research and development tasks, yet existing benchmarks typically evaluate them on prescribed workflows or n

Applications

Learning to Detect UI Principle Violations via Reinforcement Learning

arXiv:2607.20690v1 Announce Type: new Abstract: Small language models and coding agents increasingly generate web front-end code, yet their outputs are typically evaluated primarily for functional cor

applicationsarxiv-cs-cl

Position Bias is Hidden Behind Ceiling Effects: A Permutation Diagnostic for LLM Benchmarks

model-releasesarxiv-cs-cl

arXiv:2607.20864v1 Announce Type: cross Abstract: Position bias in multiple-choice LLM evaluation is widely cited as a confound in capability comparisons, but published measurements rely on single ans

Local Ai

Rethinking Open-World Video Anomaly Detection: Diagnosing Definition Blindness

arXiv:2607.20780v1 Announce Type: new Abstract: Open-world video anomaly detection (OWVAD) is expected to detect events that match a user-specified definition of abnormality. This requirement is stron

local-aiarxiv-cs-cv

Agents

SevDiff: Severity-Conditioned Diffusion for Long-Tail Conflict Trajectory Generation

arXiv:2607.20549v1 Announce Type: new Abstract: Trajectory datasets used in ADAS evaluation are heavily biased toward routine driving; genuine vehicle-to-vehicle conflict events are rare, and the rare

agentsarxiv-cs-lg

Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning

arXiv:2607.21300v1 Announce Type: cross Abstract: Machine unlearning has emerged as a tool for removing personal data from trained models to comply with recent AI regulations. To evaluate unlearning e

CrackedPDFs: A Controlled Benchmark for Hidden Prompt Injection in PDFs

arXiv:2607.19396v1 Announce Type: new Abstract: Document-based LLM systems often flatten a PDF before guardrails inspect it. That step can discard evidence that an instruction was never visible to the

Detect Early, Escalate Rarely: Anytime Detection of AI-Generated Video from the Compressed Bitstream

arXiv:2607.19476v1 Announce Type: new Abstract: Detectors for AI-generated video are evaluated offline. A clip is decoded to pixels and scored once, increasingly by a large vision-language model. Dete

researcharxiv-cs-cv

Enhancing LLMs for Identifying and Prioritizing Important Medical Jargons from Electronic Health Record Notes Utilizing Data Augmentation: A Comparative Study

model-releasesarxiv-cs-cl

arXiv:2502.16022v3 Announce Type: replace Abstract: OpenNotes gives patients access to their EHR notes, but dense medical jargon limits comprehension. We evaluate closed-source and open-source LLMs fo

Guardrails as Scapegoats: Auditing Unfaithful Safety Refusals in Tool-Augmented LLM Agents

arXiv:2607.19449v1 Announce Type: cross Abstract: Evaluation frameworks for tool-augmented LLM agents focus overwhelmingly on capability metrics or explicit tool crashes, leaving silent infrastructure

ITPEval: Benchmarking Formal Translation Across Interactive Theorem Provers

arXiv:2607.19407v1 Announce Type: new Abstract: Formal theorem proving has emerged as a frontier challenge for machine learning, yet the ecosystem is fragmented: proofs remain siloed across incompatib

Tutorials

Lightweight Person-Place Relation Extraction from Historical Newspapers with Dependency Graphs and Proximity Features

arXiv:2607.19718v1 Announce Type: new Abstract: The HIPE-2026 shared task introduces person-place relation extraction from multilingual historical newspapers as a new evaluation track, classifying the

tutorialsarxiv-cs-cl

LoRFT: Benchmarking Long-Range Vehicle Trajectory Reconstruction from Fixed Highway Cameras

arXiv:2607.19911v1 Announce Type: new Abstract: Long-range vehicle trajectories provide important spatio-temporal evidence for traffic safety analysis, autonomous driving evaluation, and data-driven t

Applications

MoAKE: Toward Unified All-in-One Action Quality Assessment via Mixture of Action Knowledge Experts

arXiv:2607.19826v1 Announce Type: new Abstract: Action Quality Assessment (AQA) aims to objectively evaluate performance quality from action videos. Most existing methods follow a ``one-by-one'' parad

applicationsarxiv-cs-cv

On the Systematic Challenges of Culturally Loaded Machine Translation: Dream of the Red Chamber as the Cultural Lens

arXiv:2607.20241v1 Announce Type: cross Abstract: Culturally loaded translation poses unique challenges for machine translation (MT), as meanings are deeply embedded in socio-cultural contexts beyond

researcharxiv-cs-ai

Outperforming Fable 5 at half the price: meet model synthesis, a new server-side tool on DigitalOcean Inference Engine

model-releasesdigitalocean

Anyone building with AI eventually hits the same tradeoff: how to get the most intelligence per dollar, the right model at the right cost for each task. That’s what DigitalOcean Inference Engine is bu

PathAgentBench: Benchmarking Evidence-Seeking Vision-Language Models on Whole-Slide Pathology Image

arXiv:2607.19261v1 Announce Type: new Abstract: Whole-slide image (WSI) diagnosis requires identifying diagnostically relevant regions, examining them across magnifications, and integrating multi-scal

PAVXploreRL: Physical-Action-Visual World Model Reinforcement Learning with Action Exploration

arXiv:2607.16602v2 Announce Type: replace Abstract: Action-conditioned world models are a key component of embodied AI, serving as scalable policy evaluators that reduce reliance on expensive real-wor

safetyarxiv-cs-cv

Refnd: Preventing Data Leakage in Relational Datasets

arXiv:2607.19376v1 Announce Type: cross Abstract: Machine learning models trained on biochemical data are routinely evaluated using splits that fail to account for relational structure, causing inform

researcharxiv-cs-lg

Stateful Guardrails for Multi-Turn LLM Systems: A Conversational Risk Accumulation Framework

arXiv:2607.19361v1 Announce Type: cross Abstract: Most safety guardrails for large language models (LLMs) evaluate each prompt-response pair in isolation, which misses failures that arise only over a

TalentCLEF at CLEF2026: Skill and Job Title Intelligence for Human Capital Management

arXiv:2607.20009v1 Announce Type: new Abstract: This paper presents the second edition of the TalentCLEF Challenge, which will run as an evaluation lab as part of CLEF 2026. The aim of TalentCLEF is t

safetyarxiv-cs-cl

Universality Reconsidered: Rethinking the Validation of Foundation Models for General-Purpose 3D Medical Segmentation

arXiv:2602.07643v2 Announce Type: replace Abstract: Foundation models have emerged as a transformative paradigm in 3D medical imaging, with the promise of unified quantitative analysis across diverse

Applications

When Benchmarks Mislead: Shortcut Learning, Length Confounds, and the Limits of Cross-Dataset Generalization in Multilingual Fake News and Sarcasm Detection

arXiv:2607.14131v2 Announce Type: replace Abstract: Cross-dataset generalisation is a fundamental requirement for deploying text classifiers in real-world settings, yet systematic evaluation across co

applicationsarxiv-cs-cl

nuTruck: Benchmarking Autonomous Driving Planning for Distributed Electric-drive Trucks

model-releasesarxiv-cs-ro

arXiv:2607.13704v1 Announce Type: new Abstract: The dominance of traditional rule-based methods in autonomous driving has gradually been replaced by learning-based approaches. While learning-based pla

16 Jul 2026

STOCKTAKE: Measuring the Gap Between Perception and Action in LLM Agents with a Fair Oracle

arXiv:2607.13618v1 Announce Type: new Abstract: LLM agents are increasingly evaluated on multi-week decision tasks in which the state that drives cost is never directly observed. On such tasks the fin

16 Jul 2026

A Biomimetic Myoelectric Tentacle Prosthesis with Sensorless Object Detection and Vibrotactile Feedback

arXiv:2607.09807v2 Announce Type: replace Abstract: This paper presents the design and evaluation of a myoelectric tentacle-shaped prosthesis integrating electromyographic (EMG) control, sensorless ob

researcharxiv-cs-ro

ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memory

model-releasesarxiv-cs-ro

arXiv:2607.10350v1 Announce Type: cross Abstract: Recent VLM and VLA systems have improved robotic perception and action prediction, yet long-horizon embodied agents still require a general runtime la

Agentic systems for breast cancer treatment recommendations

model-releasesarxiv-cs-cl

arXiv:2607.12051v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly being explored for clinical decision support, but their reliability in complex oncology treatment planning

GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents

arXiv:2607.11192v2 Announce Type: replace Abstract: A large share of day-to-day work in professional domains happens inside PDF files: benefits packets, leases, datasheets, clinical guidelines, constr

How Query Visibility Changes KV-Cache Compression Rankings: A Matched-Budget Audit

arXiv:2607.11942v1 Announce Type: cross Abstract: KV-cache compression methods are predominantly evaluated with the query appended to the context before compression -- a query-aware protocol. Yet the

researcharxiv-cs-ai

Agents

Human-AI Agent Interaction as a Neuroplastic Training Environment

arXiv:2607.12823v1 Announce Type: new Abstract: Interaction with AI agents has become one of the most frequent activities of everyday digital life. Whether conversing with an assistant, working with a

agentsarxiv-cs-ai

Learning When to Trust in Contextual Social Bandits

arXiv:2603.13356v2 Announce Type: replace Abstract: Robust reinforcement learning typically assumes that feedback sources are either globally trustworthy or corrupted within a fixed global budget. We

Lost in Visual Translation: A VLM-Assisted Perceptual-Semantic Coherence Framework for EEG-to-Image Reconstruction

arXiv:2607.12364v1 Announce Type: cross Abstract: EEG-to-image evaluation should distinguish visual fidelity from recoverable meaning. Yet EEG-derived reconstructions are blurry, distorted, and low-de

MAG: A Web-Agent Benchmark and Harness for Multimodal Action and Guide Generation

arXiv:2607.10079v2 Announce Type: replace Abstract: Digital Adoption Platforms (DAPs) are embedded overlays widely used on web systems to guide users through operations inside a page, helping them get

NEEDL-Bench: Dataset for Swiss Needle Cast and Stomata Detection in Microscopy Images

arXiv:2607.12076v1 Announce Type: new Abstract: We present NEEDL-Bench, a microscopy detection benchmark for Swiss Needle Cast (SNC), a fungal disease of Douglas-fir trees. Douglas-fir is a keystone s

RCWT: Measuring Task-Budget Displacement from Coordination Content in LLM Calls

arXiv:2607.12216v1 Announce Type: cross Abstract: Multi-agent and memory-augmented LLM systems often place coordination content, shared state, prior discussion, tool outputs, summaries, and role instr

Sample Efficient Generative Optimization for Molecular Design

model-releasesarxiv-cs-lg

arXiv:2607.12488v1 Announce Type: new Abstract: Molecular optimization in drug discovery, materials design, and catalysis requires searching vast chemical spaces under tight evaluation budgets, since

Streamlining stereo differentiable rendering for marker-free real-time tracking of surgical robots

arXiv:2607.12604v1 Announce Type: new Abstract: Purpose: Marker-based tracking of surgical robots is occlusion-prone in cluttered operating rooms. We evaluate stereo differentiable rendering for marke

safetyarxiv-cs-ro

TerraLogic: A Benchmark for Hierarchical Geospatial Reasoning in Earth Observation

arXiv:2607.12497v1 Announce Type: new Abstract: Beyond perception, reasoning is essential in remote sensing for advanced interpretation, inference, and decision-making. Recent advances in large langua

The Sound of Absence: Audio-Language Embedding Models Struggle with Negation

arXiv:2607.12290v1 Announce Type: cross Abstract: Audio-language embedding models such as CLAP are widely evaluated on matching present sound events, but rarely on negation. We show this affirmation-o

The Spectrum Is Not Enough: When Context Helps Time-Series Forecasting

arXiv:2607.13006v1 Announce Type: new Abstract: A growing family of indices scores how predictable a series is from its spectrum. Practitioners increasingly read these scores as answering a different

researcharxiv-cs-lg

Local Ai

UR-VC: Unsupervised Robotic Value Correction for Time-Derived Progress Proxies

arXiv:2607.12892v1 Announce Type: cross Abstract: Modern robot learning systems increasingly rely on dense progress or value signals to evaluate intermediate states, guide policy learning, and detect

local-aiarxiv-cs-ai

Hardware

VQCSim: When Does Compile-Once Statevector Simulation Beat Generic Quantum Frameworks?

arXiv:2607.11985v1 Announce Type: cross Abstract: Hybrid quantum-classical machine learning workflows repeatedly evaluate many small parametrized circuits during training and model exploration. In thi

hardwarearxiv-cs-lg

DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks