AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,652
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,793
  • Industry6,824
  • Local Ai7,051
  • Model Releases36,896
  • Research29,820
  • Safety19,754
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,652
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,793
  • Industry6,824
  • Local Ai7,051
  • Model Releases36,896
  • Research29,820
  • Safety19,754
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,652Total entries
60Added by human
127,592Found by agent
13Categories

Knowledge catalogue

Search: “evaluation”

GridTimelineEvolution
41,190 results
12 May 2026

Threat Modelling using Domain-Adapted Language Models: Empirical Evaluation and Insights

ResearchDGX agent

arXiv:2605.10808v1 Announce Type: cross Abstract: Large Language Models(LLMs) are increasingly explored for cybersecurity applications such as vulnerability detection. In the domain of threat modellin

11 May 2026

Multi-Dimensional Evaluation of LLMs for Grammatical Error Correction

ResearchDGX agent

arXiv:2605.07635v1 Announce Type: new Abstract: Automated assistants for Grammatical Error Correction are now embedded in educational platforms serving millions of learners, yet three critical gaps re

Q-MMR: Off-Policy Evaluation via Recursive Reweighting and Moment Matching

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Safety
DGX agent

arXiv:2605.06474v2 Announce Type: replace-cross Abstract: We present a novel theoretical framework, Q-MMR, for off-policy evaluation in finite-horizon MDPs. Q-MMR learns a set of scalar weights, one f

Robustness of Refugee-Matching Gains to Off-Policy Evaluation Choices

SafetyDGX agent

arXiv:2605.06686v1 Announce Type: new Abstract: Previous research has investigated the potential of refugee matching for boosting refugee outcomes, first considered by Bansak et al. (2018). This paper

Sources: the White House's Office of the National Cyber Director and Commerce Department's CAISI are fighting over which agency should lead AI model evaluations (Washington Post)

SafetyDGX agent

Washington Post: Sources: the White House's Office of the National Cyber Director and Commerce Department's CAISI are fighting over which agency should lead AI model evaluations — As the White House g

Unsolvability Ceiling in Multi-LLM Routing: An Empirical Study of Evaluation Artifacts

Model ReleasesDGX agent

arXiv:2605.07395v1 Announce Type: cross Abstract: Efficient routing across multiple LLMs enables cost-quality tradeoffs by directing queries to the cheapest capable model. Prior work attributes routin

VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents

AgentsDGX agent

arXiv:2506.21582v5 Announce Type: replace-cross Abstract: Text analytics has traditionally required specialized knowledge in Natural Language Processing (NLP) or text analysis, which presents a barrie

Zero-Shot Neural Network Evaluation with Sample-Wise Activation Patterns

HardwareDGX agent

arXiv:2605.07378v1 Announce Type: new Abstract: Zero-shot proxies, also known as training-free metrics, are widely adopted to reduce the computational overhead in neural network evaluation for scenari

7 May 2026

Ensuring Reliability in Programming Knowledge Tracing: A Re-evaluation of Attention-augmented Models and Experimental Protocols

ResearchDGX agent

arXiv:2605.04727v1 Announce Type: new Abstract: Programming Knowledge Tracing (PKT) has recently advanced through hybrid approaches that integrate attention-based feature modeling for code representat

Unintended Negative Impacts of Promotional Language in Patent Evaluation

ResearchDGX agent

arXiv:2605.04926v1 Announce Type: new Abstract: Promotional language has been increasingly used to aid the communication of innovative ideas in science. Yet, less is known about its role in the contex

5 May 2026

Sim-to-Real Transfer and Robustness Evaluation of Reinforcement Learning Control with Integrated Perception on an ASV for Floating Waste Capture

AgentsDGX agent

arXiv:2605.02529v1 Announce Type: new Abstract: Autonomous surface vessels for floating-waste removal operate under varying hydrodynamics, external disturbances, and challenging water-surface percepti

4 May 2026

Banger paper from Meta FAIR. They introduce Autodata, an agentic data scientist that builds high-quality training and evaluation data autono…

AgentsDGX agent

Banger paper from Meta FAIR. They introduce Autodata, an agentic data scientist that builds high-quality training and evaluation data autonomously. The headline result: on a CS research QA task, an Ag

Can Small Language Models Handle Context-Summarized Multi-Turn Customer-Service QA? A Synthetic Data-Driven Comparative Evaluation

SafetyDGX agent

arXiv:2602.00665v3 Announce Type: replace Abstract: Customer-service question answering (QA) systems increasingly rely on conversational language understanding. While Large Language Models (LLMs) achi

Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment

Model ReleasesDGX agent

arXiv:2605.00022v1 Announce Type: new Abstract: The rapid proliferation of large audio models (LAMs) demands efficient approaches for model comparison, yet comprehensive benchmarks are costly. To fill

1 May 2026

Instruction Complexity Induces Positional Collapse in Adversarial LLM Evaluation

Model ReleasesDGX agent

arXiv:2604.27249v1 Announce Type: cross Abstract: When instructed to underperform on multiple-choice evaluations, do language models engage with question content or fall back on positional shortcuts?

Qualitative Evaluation of Language Model Rescoring in Automatic Speech Recognition

ResearchDGX agent

arXiv:2604.27533v1 Announce Type: new Abstract: Evaluating automatic speech recognition (ASR) systems is a classical but difficult and still open problem, which often boils down to focusing only on th

30 Apr 2026

From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents

Model ReleasesDGX agent

arXiv:2603.25342v2 Announce Type: replace Abstract: Deep Research Agents (DRAs) aim to answer complex questions by searching the web, checking evidence, and synthesizing conclusions across heterogeneo

29 Apr 2026

Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models

SafetyDGX agent

arXiv:2512.20677v4 Announce Type: replace-cross Abstract: The increasing deployment of large language models (LLMs) in safety-critical applications raises fundamental challenges in systematically eval

PSI-Bench: Towards Clinically Grounded and Interpretable Evaluation of Depression Patient Simulators

Model ReleasesDGX agent

arXiv:2604.25840v1 Announce Type: new Abstract: Patient simulators are gaining traction in mental health training by providing scalable exposure to complex and sensitive patient interactions. Simulati

VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation

Model ReleasesDGX agent

arXiv:2604.25235v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly used as automated judges for multimodal systems, yet their scores provide no indication of reliability.

28 Apr 2026

An Empirical Evaluation of Locally Deployed LLMs for Bug Detection in Python Code

Model ReleasesDGX agent

arXiv:2604.23361v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated strong performance on a wide range of software engineering tasks, including code generation and analysi

An empirical evaluation of the risks of AI model updates using clinical data: stability, arbitrariness, and fairness

SafetyDGX agent

arXiv:2604.23954v1 Announce Type: new Abstract: Artificial Intelligence and Machine Learning (AI/ML) models used in clinical settings are increasingly deployed to support clinical decision-making. How

Expert Evaluation of LLM's Open-Ended Legal Reasoning on the Japanese Bar Exam Writing Task

ApplicationsDGX agent

arXiv:2604.23730v1 Announce Type: new Abstract: Large language models (LLMs) have shown strong performance on legal benchmarks, including multiple-choice components of bar exams. However, their capaci

Peer Identity Bias in Multi-Agent LLM Evaluation: An Empirical Study Using the TRUST Democratic Discourse Analysis Pipeline

SafetyDGX agent

arXiv:2604.22971v1 Announce Type: cross Abstract: The TRUST democratic discourse analysis pipeline exposes its large language model (LLM) components to peer model identity through multiple structural

Secure On-Premise Deployment of Open-Weights Large Language Models in Radiology: An Isolation-First Architecture with Prospective Pilot Evaluation

Model ReleasesDGX agent

arXiv:2604.22768v1 Announce Type: cross Abstract: Purpose: To design, implement, evaluate, and report on the regulatory requirements of a self-hosted LLM infrastructure for radiology adhering to the p

StratRAG: A Multi-Hop Retrieval Evaluation Dataset for Retrieval-Augmented Generation Systems

Model ReleasesDGX agent

arXiv:2604.22757v1 Announce Type: cross Abstract: We introduce StratRAG, an open-source retrieval evaluation dataset for benchmarking Retrieval-Augmented Generation (RAG) systems on multi-hop reasonin

25 Apr 2026

OpenAI just released HealthBench Professional on Hugging Face A medical evaluation benchmark designed to improve AI assistants for clinician…

Model ReleasesDGX agent

OpenAI just released HealthBench Professional on Hugging Face A medical evaluation benchmark designed to improve AI assistants for clinicians, featuring physician-curated conversations and rubric-base

24 Apr 2026

Rethinking Cross-Domain Evaluation for Face Forgery Detection with Semantic Fine-grained Alignment and Mixture-of-Experts

SafetyDGX agent

arXiv:2604.21478v1 Announce Type: new Abstract: Nowadays, visual data forgery detection plays an increasingly important role in social and economic security with the rapid development of generative mo

22 Apr 2026

A Network-Aware Evaluation of Distributed Energy Resource Control in Smart Distribution Systems

ResearchDGX agent

arXiv:2604.19715v1 Announce Type: new Abstract: Distribution networks with high penetration of Distributed Energy Resources (DERs) increasingly rely on communication networks to coordinate grid-intera

When Graph Structure Becomes a Liability: A Critical Re-Evaluation of Graph Neural Networks for Bitcoin Fraud Detection under Temporal Distribution Shift

ResearchDGX agent

arXiv:2604.19514v1 Announce Type: cross Abstract: The consensus that GCN, GraphSAGE, GAT, and EvolveGCN outperform feature-only baselines on the Elliptic Bitcoin Dataset is widely cited but has not be

21 Apr 2026

Beyond Word Boundaries: A Hebrew Coreference Benchmark and an Evaluation Protocol for Morphologically Complex Text

Model ReleasesDGX agent

arXiv:2604.17108v1 Announce Type: new Abstract: Coreference Resolution (CR) is a fundamental NLP task critical for long-form tasks as information extraction, summarization, and many business applicati

HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Models

ResearchDGX agent

arXiv:2511.01066v3 Announce Type: replace Abstract: We present an ongoing initiative to provide open, very large, high-quality, and richly annotated textual datasets for almost 200 languages. At 30 tr

Spotlights and Blindspots: Evaluation Machine-Generated Text Detection

ResearchDGX agent

arXiv:2604.16607v1 Announce Type: new Abstract: With the rise of generative language models, machine-generated text detection has become a critical challenge. A wide variety of models is available, bu

20 Apr 2026

CTSCAN: Evaluation Leakage in Chest CT Segmentation and a Reproducible Patient-Disjoint Benchmark

Model ReleasesDGX agent

arXiv:2604.15561v1 Announce Type: cross Abstract: Reported chest CT segmentation performance can be strongly inflated when train and test partitions mix slices from the same study. We present CTSCAN,

Enabling Predictive Maintenance in District Heating Substations: A Labelled Dataset and Fault Detection Evaluation Framework based on Service Data

Model ReleasesDGX agent

arXiv:2511.14791v2 Announce Type: replace-cross Abstract: Early detection of faults in district heating substations is imperative to reduce return temperatures and enhance efficiency. However, progres

17 Apr 2026

Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluation

AgentsDGX agent

arXiv:2601.07338v2 Announce Type: replace Abstract: Large Language Models (LLMs) have significantly advanced Machine Translation (MT), applying them to linguistically complex domains-such as Social Ne

Blinded Multi-Rater Comparative Evaluation of a Large Language Model and Clinician-Authored Responses in CGM-Informed Diabetes Counseling

SafetyDGX agent

arXiv:2604.15124v1 Announce Type: new Abstract: Continuous glucose monitoring (CGM) is central to diabetes care, but explaining CGM patterns clearly and empathetically remains time-intensive. Evidence

IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation

Model ReleasesDGX agent

arXiv:2511.01014v3 Announce Type: replace Abstract: Instruction-following is a fundamental ability of Large Language Models (LLMs), requiring their generated outputs to follow multiple constraints imp

15 Apr 2026

A Dataset and Evaluation for Complex 4D Markerless Human Motion Capture

SafetyDGX agent

arXiv:2604.12765v1 Announce Type: new Abstract: Marker-based motion capture (MoCap) systems have long been the gold standard for accurate 4D human modeling, yet their reliance on specialized hardware

Development, Evaluation, and Deployment of a Multi-Agent System for Thoracic Tumor Board

AgentsDGX agent

arXiv:2604.12161v1 Announce Type: new Abstract: Tumor boards are multidisciplinary conferences dedicated to producing actionable patient care recommendations with live review of primary radiology and

14 Apr 2026

FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoning

SafetyDGX agent

arXiv:2604.10693v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has improved LLM reasoning, but models often generate explanations that appear coherent while containing unfaithful int

FinTrace: Holistic Trajectory-Level Evaluation of LLM Tool Calling for Long-Horizon Financial Tasks

Model ReleasesDGX agent

arXiv:2604.10015v1 Announce Type: new Abstract: Recent studies demonstrate that tool-calling capability enables large language models (LLMs) to interact with external environments for long-horizon fin

Hidden Failures in Robustness: Why Supervised Uncertainty Quantification Needs Better Evaluation

ResearchDGX agent

arXiv:2604.11662v1 Announce Type: new Abstract: Recent work has shown that the hidden states of large language models contain signals useful for uncertainty estimation and hallucination detection, mot

Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMs

Model ReleasesDGX agent

arXiv:2604.11120v1 Announce Type: new Abstract: Personality imbuing customizes LLM behavior, but safety evaluations almost always study prompt-based personas alone. We show this is incomplete: prompti

UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agents

Model ReleasesDGX agent

arXiv:2604.11557v1 Announce Type: new Abstract: Tool-use capability is a fundamental component of LLM agents, enabling them to interact with external systems through structured function calls. However

Utilizing and Calibrating Hindsight Process Rewards via Reinforcement with Mutual Information Self-Evaluation

SafetyDGX agent

arXiv:2604.11611v1 Announce Type: new Abstract: To overcome the sparse reward challenge in reinforcement learning (RL) for agents based on large language models (LLMs), we propose Mutual Information S

13 Apr 2026

From Frames to Events: Rethinking Evaluation in Human-Centric Video Anomaly Detection

Local AiDGX agent

arXiv:2604.09327v1 Announce Type: new Abstract: Pose-based Video Anomaly Detection (VAD) has gained significant attention for its privacy-preserving nature and robustness to environmental variations.

Interactive ASR: Towards Human-Like Interaction and Semantic Coherence Evaluation for Agentic Speech Recognition

AgentsDGX agent

arXiv:2604.09121v1 Announce Type: cross Abstract: Recent years have witnessed remarkable progress in automatic speech recognition (ASR), driven by advances in model architectures and large-scale train

SAGE: A Service Agent Graph-guided Evaluation Benchmark

Model ReleasesDGX agent

arXiv:2604.09285v1 Announce Type: new Abstract: The development of Large Language Models (LLMs) has catalyzed automation in customer service, yet benchmarking their performance remains challenging. Ex

TaxPraBen: A Scalable Benchmark for Structured Evaluation of LLMs in Chinese Real-World Tax Practice

Model ReleasesDGX agent

arXiv:2604.08948v1 Announce Type: new Abstract: While Large Language Models (LLMs) excel in various general domains, they exhibit notable gaps in the highly specialized, knowledge-intensive, and legal

10 Apr 2026

Beyond Surface Judgments: Human-Grounded Risk Evaluation of LLM-Generated Disinformation

SafetyDGX agent

arXiv:2604.06820v1 Announce Type: new Abstract: Large language models (LLMs) can generate persuasive narratives at scale, raising concerns about their potential use in disinformation campaigns. Assess

16 Oct 2026

KBS: A Korean Benchmark Suite for Foundation Model Evaluation

Model ReleasesDGX agent

arXiv:2610.21227v1 Announce Type: new Abstract: We present a Korean Benchmark Suite (KBS) comprising Ko-ARC, Ko-GSM8K, Ko-EQ-Bench, Ko-WinoGrande, Ko-LAMBADA, and Ko-IFEval, covering six capabilities

MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments

AgentsDGX agent

arXiv:2610.20939v1 Announce Type: new Abstract: Online reinforcement learning (RL) with verifiable rewards has driven rapid progress in mathematics, coding, and tool use, but scaling this success to G

15 Oct 2026

A Strong Harness Narrows the Model Gap: A Comparative Evaluation for Legislative Document-to-XML Conversion

Model ReleasesDGX agent

arXiv:2610.19999v1 Announce Type: new Abstract: Legal-content reproduction requires matching the source exactly in section hierarchy, verbatim text, citations, and amendment spans; a dropped citation

14 Oct 2026

A Controlled Evaluation of Curvature Inspired Scalar Feature Scaling for Domain Generalization

ResearchDGX agent

arXiv:2610.18227v1 Announce Type: cross Abstract: Domain-specific curvature is an appealing idea for domain generalization (DG), but curvature learning can denote substantially different computations.

Auditing Observation-Quality Shortcuts in the Evaluation of Person-Centered Motion for Video Anomaly Detection

SafetyDGX agent

arXiv:2610.16471v1 Announce Type: new Abstract: Video anomaly detection (VAD) identifies deviations from normal patterns, but such deviations do not necessarily correspond to safety-critical events. W

Beyond a Better Score: Long-Horizon Agentic ML Development and Evaluation Protocol for Physics Time Series

Model ReleasesDGX agent

arXiv:2610.14024v1 Announce Type: new Abstract: Physics time-series data are central to fundamental discoveries involving dark matter, neutrinos, and gravitational waves. Machine learning has shown st

Deep Learning-based Semantic Segmentation of Dental Radiographs: Comparative Evaluation of PSPNet, DeepLabv3, UNet, and UNet++

ResearchDGX agent

arXiv:2610.15061v1 Announce Type: new Abstract: Accurate and efficient analysis of dental radiographs has become increasingly important for disease diagnosis and clinical decision making. In particula

GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation

Model ReleasesDGX agent

arXiv:2610.17794v1 Announce Type: new Abstract: Contextual forecasting seeks to improve time series forecasting by incorporating forward-looking information, such as planned events, external condition

LLM Evaluation is a Forecasting Problem

ApplicationsDGX agent

arXiv:2610.17018v1 Announce Type: new Abstract: Standard procedure in machine learning is to randomly split train and test data, emulating IID draws from one underlying data distribution. In practice,

← Previous
1…2728293031…687
Next →