Search: “evaluation”

Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One

arXiv:2606.25449v1 Announce Type: new Abstract: A language model's memory can be worse than having no memory at all. Give a model a memory that kept a wrong conclusion but dropped the work behind it,

25 Jun 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

Federated Survival Analysis in Healthcare: A Multi-Model Evaluation on Cross-Institutional Heterogeneous Breast Cancer Data

DGX agent

arXiv:2606.23871v1 Announce Type: new Abstract: Survival analysis is central to clinical decision-making, yet reliable time-to-event models require large, diverse cohorts that are rarely available at

local-aiarxiv-cs-lg

24 Jun 2026

Agents

Managing Task Execution for Unknown Workloads in Batteryless IoT: A Hardware-Agnostic Evaluation

arXiv:2606.24340v1 Announce Type: new Abstract: In recent years, the Internet of Things (IoT) paradigm has been shifting toward batteryless, energy-harvesting architectures. Sustaining reliable operat

agentsarxiv-cs-lg

24 Jun 2026

T2D-Bench: Evidence-Gated Evaluation of LLM Outputs for Type 2 Diabetes Using a Multi-Layer Clinical-Lifestyle Knowledge Graph

arXiv:2606.24145v1 Announce Type: new Abstract: Large language models (LLMs) can produce clinically fluent recommendations for type 2 diabetes while failing to satisfy guideline constraints or explici

24 Jun 2026

Can LLMs Control Readability? A Multi-Dimensional Evaluation Framework for CEFR-Controlled Arabic Generation

arXiv:2606.21981v1 Announce Type: cross Abstract: While Large Language Models (LLMs) can generate fluent Arabic text, their ability to reliably control readability levels remains unclear. We propose a

safetyarxiv-cs-lg

Chains That See, Answers That Don't: A Multi-Aspect Evaluation Recipe for Forced Chain-of-Thought on Video-MME

model-releasesarxiv-cs-cv

arXiv:2606.22862v1 Announce Type: new Abstract: Forced chain-of-thought (CoT) is widely assumed to make vision-language models more reliable on video question answering. We propose a small three-probe

Comparative Evaluation of Machine Learning and Deep Learning Models for Wound-Rotor Synchronous Motor Performance Prediction

model-releasesarxiv-cs-lg

arXiv:2606.21230v1 Announce Type: new Abstract: Wound rotor synchronous motors have emerged as a strong alternative that eliminates dependence on REEs. However, WRSM design requires the simultaneous o

Fed-CausalDiff: Decoupled Synchronization for Federated Do-Simulation and Policy Evaluation

arXiv:2606.22510v1 Announce Type: new Abstract: While federated learning enables collaborative modelling on decentralised data, standard methods merely fit historical observations. This purely observa

local-aiarxiv-cs-lg

BioDivergence: A Benchmark and Evaluation Framework for Hidden Contextual Contradictions in Biomedical Abstracts

arXiv:2606.11208v1 Announce Type: cross Abstract: Biomedical findings often seem to conflict across studies, but many of these differences are context-dependent rather than true contradictions. Variat

11 Jun 2026

Frozen Foundation-Model Embeddings Discard Small-Lesion Signal in Chest Radiography: Implications for Pre-Deployment Evaluation

arXiv:2606.11606v1 Announce Type: new Abstract: Frozen vision-transformer (ViT) foundation-model embeddings increasingly serve as the substrate for downstream chest-radiography (CXR) pipelines, yet wh

local-aiarxiv-cs-cv

11 Jun 2026

A complementary study on PlanGPT: Evaluation with defined Performance Metrics and comparison with a planner

arXiv:2606.10489v1 Announce Type: new Abstract: Automated Planning is a subfield of Artificial Intelligence (AI) where the main objective is generating a sequence of actions, known as a plan, that hel

Recalling Too Well: Sycophancy Evaluation and Mitigation in Memory-Augmented Models

arXiv:2606.10949v1 Announce Type: new Abstract: Persistent memory systems promise to make LLMs more helpful by storing user beliefs over time. We show they also make models less correct by systematica

What Really Matters for Table LLMs? A Meta-Evaluation of Model and Data Effects

arXiv:2501.14717v2 Announce Type: replace Abstract: Table modeling has progressed for decades. In this work, we revisit this trajectory and highlight emerging challenges in the LLM era, particularly t

researcharxiv-cs-cl

Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields

arXiv:2606.11042v1 Announce Type: new Abstract: Recent years have witnessed the rapid evolution of AI agents toward handling increasingly complex, real-world tasks. However, existing benchmarks rarely

Comparative evaluation of training strategies using partially labelled datasets for segmentation of white matter hyperintensities and stroke lesions in FLAIR MRI

arXiv:2601.20503v2 Announce Type: replace-cross Abstract: White matter hyperintensities (WMH) and ischaemic stroke lesions (ISL) are key imaging biomarkers of cerebral small vessel disease (SVD) detec

safetyarxiv-cs-ai

Applications

Cybernetic Android Avatar 'Yui': System Integration, Field Deployment, and Evaluation

arXiv:2606.08099v1 Announce Type: new Abstract: Remote communication technologies have become widely used; however, supporting a sense of shared physical space and conveying rich non-verbal cues remai

applicationsarxiv-cs-ro

Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics

arXiv:2606.08417v1 Announce Type: cross Abstract: Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling. Progress in both

Multimodal Large Language Models as Synthetic Participants in Video-Based Studies: An Evaluation

arXiv:2606.07541v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have shown strong performance on objective tasks such as video understanding and reasoning. However, it remai

Position: Genomic Model Research Must Move Beyond Anecdotal Evaluation of Interpretability Methods

arXiv:2606.07607v1 Announce Type: new Abstract: Advances in machine learning and computational power have unlocked the predictive potential of the human genome, yet biologists now demand that these mo

local-aiarxiv-cs-lg

Applications

Selecting New Measurement Locations to Diversify Traffic-Pattern Coverage: A Real-World Evaluation for Total Traffic Volume Estimation

arXiv:2606.07556v1 Announce Type: cross Abstract: Accurate measurement of traffic volumes and flows is vital for modern intelligent transportation. However, despite recent technological advances in se

applicationsarxiv-cs-ai

Decision-Aware Evaluation of Physics-Informed Surrogates

model-releasesarxiv-cs-lg

arXiv:2606.07146v1 Announce Type: new Abstract: Physics-informed machine learning is often assessed by curve error, although engineering use depends on downstream decisions: ranking candidates, avoidi

8 Jun 2026

LLM-Augmented Digital Twin for Policy Evaluation in Short-Video Platforms

arXiv:2603.11333v2 Announce Type: replace Abstract: Short-video platforms are closed-loop, human-in-the-loop ecosystems where platform policy, creator incentives, and user behavior co-evolve. This fee

safetyarxiv-cs-ai

8 Jun 2026

Dimensionality Reduction for Cyberattack Classification: A Comparative Evaluation of PCA and Linear Predictive Coding

arXiv:2606.05584v1 Announce Type: cross Abstract: High-dimensional feature representations are widely used in machine learning-based cyberattack detection systems. However, they increase computational

6 Jun 2026

Evaluation of LLMs for Mathematical Formalization in Lean

arXiv:2606.05632v1 Announce Type: new Abstract: Within the past few years, the ability of Large Language Models (LLMs) to generate formal mathematical proofs has improved drastically. We provide a com

6 Jun 2026

Geographic Bias and Diversity in AI Evaluation

arXiv:2606.05187v1 Announce Type: cross Abstract: Among the many challenges hindering the responsible development and deployment of AI, arguably none has faced more intense scrutiny than bias in its v

6 Jun 2026

CLEAR: Cognition and Latent Evaluation for Adaptive Routing in End-to-End Autonomous Driving

model-releasesarxiv-cs-ro

arXiv:2606.06219v1 Announce Type: new Abstract: End-to-end autonomous driving models often struggle to balance multi-modal maneuver generation with real-time inference constraints. While diffusion mod

5 Jun 2026

A Systematic Evaluation of Positional Bias in Multi-Video Summarization with MLLMs

arXiv:2606.04596v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) are increasingly used for video understanding, yet their reliability under multi-video inputs remains poorly un

4 Jun 2026

Safety Under Scaffolding: How Evaluation Conditions Shape Measured Safety

arXiv:2603.10044v2 Announce Type: replace-cross Abstract: A safety score earned on a benchmark need not predict how the same model behaves once it is wrapped in an agentic scaffold the benchmark never

4 Jun 2026

Acceptance-Test-Driven Evaluation Protocols for Business-Centric LLM Systems

arXiv:2606.02755v1 Announce Type: cross Abstract: Large language model (LLM) applications are increasingly expected to satisfy deterministic institutional requirements while relying on probabilistic g

Critical evaluation of PINN for FWD inverse analysis and differentiable FEM as an alternative

model-releasesarxiv-cs-lg

arXiv:2606.03210v1 Announce Type: cross Abstract: Automatic-differentiation-based inverse analysis methods, including physics-informed neural networks (PINNs) and differentiable programming, have rece

WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts

arXiv:2606.03220v1 Announce Type: cross Abstract: Existing benchmarks for MLLM-generated web artifacts assess interaction through local evidence and miss the requirement-induced states and transitions

local-aiarxiv-cs-ai

WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation

arXiv:2503.07265v4 Announce Type: replace-cross Abstract: Text-to-Image (T2I) models are capable of generating high-quality artistic creations and visual content. However, existing research and evalua

Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why

arXiv:2606.00093v1 Announce Type: new Abstract: Validating an LLM judge against human annotations usually means reporting several agreement statistics: accuracy, precision, recall, F_1, Cohen's kappa,

researcharxiv-cs-cl

Chunking Methods on Retrieval-Augmented Generation - Effectiveness Evaluation Against Computational Cost and Limitations

arXiv:2606.00881v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) has demonstrated significant capabilities in enhancing the performance of Large Language Models (LLMs). One of the

researcharxiv-cs-cl

Evaluation of Baseline Methods for IDD-based SSD External Memory Search

arXiv:2606.01840v1 Announce Type: new Abstract: Many difficult search problems cannot be solved by algorithms such as A* using only RAM. Search algorithms which use external memory such as SSDs and HD

On the Evaluation of Spiking Neural Network Configurations for Network Intrusion Detection

arXiv:2606.01442v1 Announce Type: cross Abstract: Network intrusion detection is a core component of modern cybersecurity infrastructure, yet the deep learning models that dominate the field are compu

Rethinking Evaluation Paradigms in IBP-based Certified Training

arXiv:2606.02134v1 Announce Type: cross Abstract: Deep neural networks achieve strong performance on many supervised learning tasks but remain vulnerable to adversarial perturbations. Neural network v

Through the PRISM: Principle-Aware, Interpretable, and Multi-Scale Evaluation of Visual Designs

model-releasesarxiv-cs-cv

arXiv:2606.00592v1 Announce Type: new Abstract: Effective visual communication stems from the harmony of multiple design principles, such as readability, contrast, alignment, overlap, and coherence, w

Tutorials

When Tabular Foundation Models Transfer Across Modalities: A Systematic Evaluation Across 95 Datasets, 7 Modalities, and Two Regimes

arXiv:2606.02106v1 Announce Type: new Abstract: We present a single classification pipeline that combines an Equiangular Tight Frame (ETF) preprocessing stage with a tabular foundation model for in-co

tutorialsarxiv-cs-lg

Design and Evaluation of Multi-Agent AI Oracle Systems for Prediction Market Resolution

arXiv:2605.30802v1 Announce Type: cross Abstract: Prediction markets aggregate collective intelligence to forecast uncertain events, but their utility depends on reliable outcome resolution. Existing

1 Jun 2026

Beyond English and Evasion: A Human-Annotated Multi-Domain Benchmark for High-Stakes LLM Safety Evaluation in Chinese

arXiv:2605.29667v1 Announce Type: new Abstract: When Large Language Models (LLMs) are deployed in Chinese-language settings, a troubling pattern emerges: safety systems that work well in English break

Bridging Chemists and AI: An Expert-Augmented Framework for Interpretable Route Evaluation

arXiv:2605.29108v1 Announce Type: new Abstract: Selecting efficient multi-step synthetic routes is a central challenge in organic synthesis, particularly in medicinal and process chemistry, where rout

researcharxiv-cs-lg

Is Your Diffusion Sampler Actually Correct? A Sampler-Centric Evaluation of Discrete Diffusion Language Models

arXiv:2602.19619v2 Announce Type: replace Abstract: Discrete diffusion language models (dLLMs) provide a fast and flexible alternative to autoregressive models (ARMs) via iterative denoising with para

researcharxiv-cs-lg

Applications

Prediction-Powered Inference Across Many Tasks for AI Evaluation & Social Science Research

arXiv:2605.29249v1 Announce Type: cross Abstract: Many applications require statistically valid inference across many related tasks, while using only a handful of high-quality labels per hypothesis. I

applicationsarxiv-cs-lg

Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation

arXiv:2605.29430v1 Announce Type: new Abstract: Automatic speech recognition (ASR) is a core component of human--computer interaction and an increasingly important front-end for LLM-based assistants a

safetyarxiv-cs-ai

A Systematic Evaluation of Retrieval-Augmented Generation and Language Models for Space Operations

arXiv:2605.27444v1 Announce Type: cross Abstract: The rapid expansion of space activities has led to an unprecedented accumulation of technical documentation, operational guidelines, and scientific li

28 May 2026

Adaptive Cost-Efficient Evaluation for Reliable Patent Claim Generation

arXiv:2604.04295v3 Announce Type: replace Abstract: Automated patent claim validation demands low error tolerance. However, existing approaches face a rigidity-resource dilemma: lightweight encoders c

28 May 2026

Chance-Constrained MPPI under State and Dynamic Object Prediction Uncertainty and the Evaluation of Collision Risk Calibration