AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “evaluation”

GridTimelineEvolution
41,190 results
27 Apr 2026

Can QPP Choose the Right Query Variant? Evaluating Query Variant Selection for RAG Pipelines

ResearchDGX agent

arXiv:2604.22661v1 Announce Type: cross Abstract: Large Language Models (LLMs) have made query reformulation ubiquitous in modern retrieval and Retrieval-Augmented Generation (RAG) pipelines, enabling

Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents

AgentsDGX agent

arXiv:2604.22452v1 Announce Type: new Abstract: Collective intelligence refers to the ability of a group to achieve outcomes beyond what any individual member can accomplish alone. As large language m

24 Apr 2026

EngramaBench: Evaluating Long-Term Conversational Memory with Structured Graph Retrieval

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Model ReleasesDGX agent

arXiv:2604.21229v1 Announce Type: cross Abstract: Large language model assistants are increasingly expected to retain and reason over information accumulated across many sessions. We introduce Engrama

Evaluating Post-hoc Explanations of the Transformer-based Genome Language Model DNABERT-2

ResearchDGX agent

arXiv:2604.21690v1 Announce Type: new Abstract: Explaining deep neural network predictions on genome sequences enables biological insight and hypothesis generation-often of greater interest than predi

From Codebooks to VLMs: Evaluating Automated Visual Discourse Analysis for Climate Change on Social Media

Model ReleasesDGX agent

arXiv:2604.21786v1 Announce Type: new Abstract: Social media platforms have become primary arenas for climate communication, generating millions of images and posts that - if systematically analysed -

Musical Score Understanding Benchmark: Evaluating Large Language Models' Comprehension of Complete Musical Scores

Model ReleasesDGX agent

arXiv:2511.20697v4 Announce Type: replace-cross Abstract: Understanding complete musical scores entails integrated reasoning over pitch, rhythm, harmony, and large-scale structure, yet the ability of

23 Apr 2026

Evaluating Black-Box Vulnerabilities with Wasserstein-Constrained Data Perturbations

SafetyDGX agent

arXiv:2603.15867v2 Announce Type: replace Abstract: The growing use of Machine Learning (ML) tools comes with critical challenges, such as limited model explainability. We propose a global explainabil

What Makes a Good AI Review? Concern-Level Diagnostics for AI Peer Review

SafetyDGX agent

arXiv:2604.19998v1 Announce Type: new Abstract: Evaluating AI-generated reviews by verdict agreement is widely recognized as insufficient, yet current alternatives rarely audit which concerns a system

22 Apr 2026

Do LLMs Game Formalization? Evaluating Faithfulness in Logical Reasoning

Model ReleasesDGX agent

arXiv:2604.19459v1 Announce Type: new Abstract: Formal verification guarantees proof validity but not formalization faithfulness. For natural-language logical reasoning, where models construct axiom s

MORPHOGEN: A Multilingual Benchmark for Evaluating Gender-Aware Morphological Generation

Model ReleasesDGX agent

arXiv:2604.18914v1 Announce Type: cross Abstract: While multilingual large language models (LLMs) perform well on high-level tasks like translation and question answering, their ability to handle gram

21 Apr 2026

Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidence

SafetyDGX agent

arXiv:2601.11886v2 Announce Type: replace Abstract: In high-stakes domains like medicine, it may be generally desirable for models to faithfully adhere to the context provided. But what happens if the

LQM: Linguistically Motivated Multidimensional Quality Metrics for Machine Translation

ResearchDGX agent

arXiv:2604.18490v1 Announce Type: new Abstract: Existing MT evaluation frameworks, including automatic metrics and human evaluation schemes such as Multidimensional Quality Metrics (MQM), are largely

ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows

Model ReleasesDGX agent

arXiv:2505.19897v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have extended their impact beyond Natural Language Processing, substantially fostering the development of interdi

Tape: A Cellular Automata Benchmark for Evaluating Rule-Shift Generalization in Reinforcement Learning

Model ReleasesDGX agent

arXiv:2601.04695v2 Announce Type: replace-cross Abstract: Out-of-distribution generalization in reinforcement learning is hard to diagnose when benchmark shifts mix dynamics, observations, goals, and

20 Apr 2026

Evaluating LLMs as Human Surrogates in Controlled Experiments

ResearchDGX agent

arXiv:2604.15329v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to simulate human responses in behavioral research, yet it remains unclear when LLM-generated data

RoleConflictBench: A Benchmark of Role Conflict Scenarios for Evaluating LLMs' Contextual Sensitivity

Model ReleasesDGX agent

arXiv:2509.25897v2 Announce Type: replace-cross Abstract: People often encounter role conflicts -- social dilemmas where the expectations of multiple roles clash and cannot be simultaneously fulfilled

15 Apr 2026

Growing Pains: Extensible and Efficient LLM Benchmarking Via Fixed Parameter Calibration

Model ReleasesDGX agent

arXiv:2604.12843v1 Announce Type: new Abstract: The rapid release of both language models and benchmarks makes it increasingly costly to evaluate every model on every dataset. In practice, models are

Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems

Model ReleasesDGX agent

arXiv:2603.01045v2 Announce Type: replace-cross Abstract: Large language models are increasingly deployed in multi-agent systems to overcome context limitations by distributing information across agen

Why Did Apple Fall: Evaluating Curiosity in Large Language Models

TutorialsDGX agent

arXiv:2510.20635v2 Announce Type: replace-cross Abstract: Curiosity serves as a pivotal conduit for human beings to discover and learn new knowledge. Recent advancements of large language models (LLMs

WiseOWL: A Methodology for Evaluating Ontological Descriptiveness and Semantic Correctness for Ontology Reuse and Ontology Recommendations

SafetyDGX agent

arXiv:2604.12025v1 Announce Type: new Abstract: The Semantic Web standardizes concept meaning for humans and machines, enabling machine-operable content and consistent interpretation that improves adv

14 Apr 2026

Do Thought Streams Matter? Evaluating Reasoning in Gemini Vision-Language Models for Video Scene Understanding

Model ReleasesDGX agent

arXiv:2604.11177v1 Announce Type: new Abstract: We benchmark how internal reasoning traces, which we call thought streams, affect video scene understanding in vision-language models. Using four config

Evaluating Visual Prompts with Eye-Tracking Data for MLLM-Based Human Activity Recognition

ResearchDGX agent

arXiv:2604.09585v1 Announce Type: cross Abstract: Large Language Models (LLMs) have emerged as foundation models for IoT applications such as human activity recognition (HAR). However, directly applyi

GazeVaLM: A Multi-Observer Eye-Tracking Benchmark for Evaluating Clinical Realism in AI-Generated X-Rays

Model ReleasesDGX agent

arXiv:2604.11653v1 Announce Type: new Abstract: We introduce GazeVaLM, a public eye-tracking dataset for studying clinical perception during chest radiograph authenticity assessment. The dataset compr

Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation

Model ReleasesDGX agent

arXiv:2604.11290v1 Announce Type: new Abstract: Synthesizing supervised finetuning (SFT) data from language models (LMs) to teach smaller models multilingual tasks has become increasingly common. Howe

Who Wrote This Line? Evaluating the Detection of LLM-Generated Classical Chinese Poetry

Model ReleasesDGX agent

arXiv:2604.10101v1 Announce Type: new Abstract: The rapid development of large language models (LLMs) has extended text generation tasks into the literary domain. However, AI-generated literary creati

Why Don't You Know? Evaluating the Impact of Uncertainty Sources on Uncertainty Quantification in LLMs

ApplicationsDGX agent

arXiv:2604.10495v1 Announce Type: new Abstract: As Large Language Models (LLMs) are increasingly deployed in real-world applications, reliable uncertainty quantification (UQ) becomes critical for safe

10 Apr 2026

Evaluating Low-Light Image Enhancement Across Multiple Intensity Levels

Model ReleasesDGX agent

arXiv:2511.15496v2 Announce Type: replace Abstract: Imaging in low-light environments is challenging due to reduced scene radiance, which leads to elevated sensor noise and reduced color saturation. M

15 Oct 2026

From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationship

Model ReleasesDGX agent

arXiv:2506.23101v2 Announce Type: replace-cross Abstract: Ensuring the trustworthiness of Multimodal Large Language Models (MLLMs) requires rigorous evaluation of gender bias, particularly in socially

Jev-ToM: Evaluating Theory of Mind in Typed Decision Models

Model ReleasesDGX agent

arXiv:2610.18610v1 Announce Type: cross Abstract: Dialogue and collaborative assistants repeatedly judge what a partner knows, believes, or intends. Typed decision models offer a potential alternative

Specification-Based Testing of Large Language Models

Model ReleasesDGX agent

arXiv:2610.14443v1 Announce Type: cross Abstract: Model cards, system cards, and safety reports increasingly document LLM safety behaviors, yet safety evaluations typically start from predefined bench

14 Oct 2026

AMemOps: Evaluating Memory Updates in Long Audio Streams

Model ReleasesDGX agent

arXiv:2610.18652v1 Announce Type: cross Abstract: Audio memory must recognise recurring sources across acoustic changes while keeping their attributes up to date. Overall accuracy does not reveal how

ChangeT^3: Evaluating Vision Test-Time Training for Remote Sensing Change Detection

Model ReleasesDGX agent

arXiv:2610.15813v1 Announce Type: new Abstract: Bitemporal remote sensing change detection requires representations that preserve spatial detail while capturing contextual information relevant to chan

Deploy and evaluate models with Amazon SageMaker JumpStart

TutorialsDGX agent

Learn how to deploy an available model from Amazon SageMaker JumpStart, run a lightweight and repeatable performance test using the SageMaker Python SDK, and delete the endpoint when testing is comple

EviScene: Beyond 'Is It AI?' for Agentic Image Factuality Assessment

Model ReleasesDGX agent

arXiv:2610.14609v1 Announce Type: new Abstract: Determining whether an image is AI-generated does not establish whether the situation it depicts is factually supported. We formulate Image Factuality A

Predict Before You Modify: Learning Improvement Dynamics for Self-Evolving Agents

AgentsDGX agent

arXiv:2610.13991v1 Announce Type: new Abstract: Self-evolving agents improve through repeated cycles of modification and evaluation. However, existing methods receive feedback on modifications only af

REEL: Learning to Retrieve, Evaluate, and Evolve Visual Memory for Any Long-Horizon Multi-Shot Video Generation

SafetyDGX agent

arXiv:2610.16632v1 Announce Type: new Abstract: Memory-based long-video generation aims to combine user control with long-range visual consistency. We organize this process into three functional compo

12 Oct 2026

From Structure to Validity: Mechanism-Guided Generation and Refinement of LLM Rubrics

ResearchDGX agent

arXiv:2610.13375v1 Announce Type: new Abstract: Rubrics have been widely used to evaluate and optimize language models on open-ended tasks. However, existing work primarily focuses on generating bette

Mind the Temporal Gap: Evaluating Video Models for Temporal Understanding in Driving

SafetyDGX agent

arXiv:2610.12575v1 Announce Type: cross Abstract: Modern video-language models are increasingly treated as general-purpose video reasoners, yet it remains unclear whether they robustly capture tempora

9 Oct 2026

BrickBench: Evaluating Agentic Brick Design

Model ReleasesDGX agent

arXiv:2610.12452v1 Announce Type: new Abstract: We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not

8 Oct 2026

From Retrieval to Customer Context: Evaluating Frontier-Model Systems for Voice-of-Customer Analysis

AgentsDGX agent

arXiv:2610.09375v1 Announce Type: new Abstract: Organizations increasingly use frontier language models to analyze customer feedback, but answer quality also depends on how that feedback is organized

Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression

Model ReleasesDGX agent

arXiv:2610.09788v1 Announce Type: new Abstract: Reward modeling often requires jointly representing and reasoning over multiple evaluation criteria, yet verbalizing this process token by token can inc

7 Oct 2026

CLM-as-a-Judge: Evaluating an Open Contrastive Decision Model on Public Judge Benchmarks

Model ReleasesDGX agent

arXiv:2610.07177v1 Announce Type: cross Abstract: An open contrastive decision model is near chance as a judge on the hard public benchmarks: Contrastive-LM/CLM-v0.1-8B scores between 0.351 (best- of-

Evaluate the Stack, Not the Layer: Do Deterministic and LLM Gates for Agent Actions Fail Independently?

AgentsDGX agent

arXiv:2610.07359v1 Announce Type: new Abstract: Runtime gates for agent tool calls are stacked on the assumption that their errors multiply. We test it on 1,119 labelled agent actions from three corpo

From Laboratory to Road: Evaluating Wearable Gaze Accuracy for Driving

SafetyDGX agent

arXiv:2610.07783v1 Announce Type: new Abstract: Bird's-eye-view (BEV) representations have become a widely used interface between perception and planning in autonomous driving, but they encode what is

4 Oct 2026

Wiki Lint Report — 2026-10-04

SynthesesDGX agent

Automated lint: 19 errors, 64915 warnings, 3 info

1 Oct 2026

Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts

Model ReleasesDGX agent

arXiv:2609.38600v1 Announce Type: new Abstract: As large language models (LLMs) are increasingly used in clinical settings, it is critical to evaluate their reliability under realistic variation in cl

Structure vs. Chain-of-Thought: Evaluating LLM Criteria Extraction for Depression Severity

ResearchDGX agent

arXiv:2609.39049v1 Announce Type: cross Abstract: A large language model (LLM) can rate depression severity directly from a social media post or mark which clinical criteria the post shows and let cod

30 Sept 2026

Evaluating Alignment of Behavioral Dispositions in LLMs

SafetyDGX agent

arXiv:2602.11328v2 Announce Type: replace Abstract: As people turn to LLMs for social advice, understanding their behavior in such contexts becomes essential. In this work, we focus on behavioral disp

Evaluating Name-Only Directory Routing for One-Shot Code Search

ResearchDGX agent

arXiv:2609.35918v1 Announce Type: cross Abstract: Finding the right files is an early challenge for coding agents. We test whether a language model can follow directory and file names to find annotate

29 Sept 2026

Evaluating Single and Multi-Omics Based Explainable Artificial Intelligence (MOXAI) for Molecular Subclass Classification of Adult-Type Diffuse Gliomas

ResearchDGX agent

arXiv:2609.32190v1 Announce Type: new Abstract: DNA methylation (DNAM) profiling has emerged as a powerful diagnostic tool for classifying brain and solid tumors. However, existing computational model

When VLMs Trust Context: Evaluating Scene Text Recognition under Misleading Context

Model ReleasesDGX agent

arXiv:2609.34781v1 Announce Type: new Abstract: Vision-language models (VLMs) can read text in natural scenes, but their predictions may be influenced by the surrounding context. When the printed text

28 Sept 2026

Efficient Safety Benchmarking via Item Response Theory

Model ReleasesDGX agent

arXiv:2606.20626v2 Announce Type: replace-cross Abstract: Safety benchmarks for language models are typically evaluated using static paradigms that treat all items as equally informative for all model

24 Sept 2026

Exact Feedback Is Not Control: Evaluating Text-based Closed-Loop Revision in LLMs

ResearchDGX agent

arXiv:2609.28150v1 Announce Type: new Abstract: Closed-loop revision is increasingly used in large language model (LLM) applications, but failures may reflect incomplete feedback or ineffective respon

23 Sept 2026

A guide to speeding up your video processing with AlphaEvolve

Model ReleasesDGX agent

In real-time streaming, every millisecond counts. For example, at 30 frames per second (fps), developers have a strict frame budget of just 33.3 ms (and only 16.6 ms at 60 fps) to ingest camera frames

Text, Pixels, or Both? Evaluating Input Representations for Multimodal Document QA

ResearchDGX agent

arXiv:2609.22628v1 Announce Type: new Abstract: Every document QA system begins with a choice that is rarely studied on its own: whether to feed the model page images, extracted text, or both. We isol

18 Sept 2026

Evaluating Explanation Methods by the Predictors They Induce

Local AiDGX agent

arXiv:2609.20058v1 Announce Type: new Abstract: Explanations of machine learning models are usually judged by criteria that are hard to compare. We propose a simpler test: if an explanation really des

When Hiring Becomes Agent-Mediated: Evaluating Access and Recurrence in Two-Agent Resume Screening

Model ReleasesDGX agent

arXiv:2609.19530v1 Announce Type: new Abstract: Hiring is bilateral: employers assess fit, while candidates present and defend evidence of their qualifications. Yet resume screening, the first gate, i

17 Sept 2026

StableEval Arena: A Cost-Aware Agentic Benchmark for Stablecoin Price Stability Prediction

Model ReleasesDGX agent

arXiv:2609.18949v1 Announce Type: cross Abstract: We introduce StableEval Arena, a cost-aware benchmark framework for evaluating agentic AI systems on stablecoin peg-risk prediction. StableEval Arena

16 Sept 2026

Evaluating Verified Autonomy in Quantum Engineering

Model ReleasesDGX agent

arXiv:2609.17439v1 Announce Type: cross Abstract: Reliable quantum engineering is essential for turning quantum phenomena into practical technologies. As quantum platforms grow in scale and complexity

15 Sept 2026

AppliedScientist: Automated Scientific Revision Through Iterative AI Reviewing

SafetyDGX agent

arXiv:2609.14738v1 Announce Type: new Abstract: Automated reviewing systems are increasingly evaluated based on the quality of the reviews they produce. Yet a review is only useful if acting on it lea

← Previous
1…6869707172…687
Next →