AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “evaluation”

GridTimelineEvolution
41,190 results
14 Sept 2026

PA-CDM: Position-Aware Character Detection Matching for Evaluating Handwritten Mathematical Expression Recognition

Model ReleasesDGX agent

arXiv:2609.12917v1 Announce Type: cross Abstract: Handwritten mathematical expression recognition (HMER) is conventionally scored by exact-match rates and string-similarity metrics that are blind to w

12 Sept 2026

Defining AI Agents: A Compendium of Criteria, Metrics, and Benchmarks

AgentsDGX agent

arXiv:2609.11018v1 Announce Type: new Abstract: The term agent in artificial intelligence lacks a standard definition, complicating the evaluation, comparison, and reproducibility of AI agent research

Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Training

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
AgentsDGX agent

arXiv:2609.10939v1 Announce Type: cross Abstract: Clinical education must prepare medical students to conduct safe and coherent patient interviews under conditions of uncertainty. Traditional standard

Finishing the Task Is Not Enough: Evaluating Agent Resilience and Considerate Participation under Accumulating Challenge

AgentsDGX agent

arXiv:2609.10724v1 Announce Type: new Abstract: Sustained deployment of generative AI agents requires more than isolated task success. Agents must remain useful across repeated interactions, changing

11 Sept 2026

Inverse Turing Bench: Evaluating Language Models as Judges of Human vs. AI Dialogue

Model ReleasesDGX agent

arXiv:2606.21844v2 Announce Type: replace Abstract: As AI systems integrate into online spaces, differentiating them from humans in conversations is increasingly important. We present Inverse Turing B

Shedding Light: A Benchmark for Evaluating Lighting Understanding in Generative Image Models

Model ReleasesDGX agent

arXiv:2609.10787v1 Announce Type: new Abstract: Accurate modelling of illumination is central to realistic image synthesis and scene understanding. Yet, there is little exploration into whether image

10 Sept 2026

From Landslide Conditioning Factors to Satellite Embeddings: Evaluating the Utilisation of Google AlphaEarth for Landslide Susceptibility Mapping using Deep Learning

ResearchDGX agent

arXiv:2601.07268v2 Announce Type: replace Abstract: Data-driven landslide susceptibility mapping (LSM) typically relies on landslide conditioning factors (LCFs), whose availability, heterogeneity, and

MetroLLM-Bench: Evaluating Language Models as Transit Kiosk Runtimes

Model ReleasesDGX agent

arXiv:2609.10016v1 Announce Type: cross Abstract: We introduce MetroLLM-Bench, a 955-case benchmark for testing language models as the policy layer of a transit kiosk. It covers six real metro systems

YallaMorph: A Benchmark for Evaluating Arabic Morphological Generation in Large Language Models

Model ReleasesDGX agent

arXiv:2609.10153v1 Announce Type: new Abstract: Arabic morphology remains challenging for large language models, since fluent generation does not guarantee accurate morphosyntactic control. Existing A

9 Sept 2026

Contextual Observer Grounding: Evaluating Situated Spatial Reasoning in Vision-Language Models

Model ReleasesDGX agent

arXiv:2609.06880v1 Announce Type: new Abstract: Reasoning over language instructions in embodied tasks such as robotics often requires understanding spatial relations from a speaker's situated perspec

Wiki Lint Report — 2026-09-09

SynthesesDGX agent

Automated lint: 12 errors, 53990 warnings, 3 info

TimeWarp: Evaluating Web Agents by Revisiting the Past

Model ReleasesDGX agent

arXiv:2603.04949v2 Announce Type: replace-cross Abstract: As web agents close the gap with humans on benchmarks, one question arises: Do today's agents perform just as well on tomorrow's web? We intro

7 Sept 2026

CC-Mediation: Evaluating Large Language Models for Cross-Cultural Conflict Mediation

Model ReleasesDGX agent

arXiv:2609.04855v1 Announce Type: cross Abstract: Cross-cultural mediation by large language models (LLMs) requires deciding both when to intervene and how to respond in culturally grounded conflicts.

How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions

AgentsDGX agent

arXiv:2609.05037v1 Announce Type: cross Abstract: As LLMs take on roles requiring moral advice, understanding how they attribute moral agency becomes critical. Humans possess moral agency, the capacit

TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors

Model ReleasesDGX agent

arXiv:2609.05117v1 Announce Type: cross Abstract: Current LLM safety benchmarks largely rely on binary metrics, overlooking how models respond to harmful prompts with varying threat implicitness. We i

4 Sept 2026

DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents

SafetyDGX agent

arXiv:2609.03423v1 Announce Type: new Abstract: Full-duplex voice agents must continuously decide when to listen, backchannel, interrupt, handle speech overlaps, take the floor, and yield. Existing be

3 Sept 2026

Evaluating Fundus-Specific Foundation Models for Diabetic Macular Edema Detection

ResearchDGX agent

arXiv:2510.07277v2 Announce Type: replace Abstract: Diabetic Macular Edema (DME) is a leading cause of vision loss among patients with Diabetic Retinopathy (DR). While deep learning has shown promisin

From Proxy Learning to Driving Decisions: A Transfer-Based Framework for Evaluating Future-Aware Autonomous Driving Planners

SafetyDGX agent

arXiv:2609.02688v1 Announce Type: new Abstract: Future-aware representations and world models are increasingly used in proposal-based autonomous-driving planners to improve trajectory selection. Howev

2 Sept 2026

A Network Science Perspective on Evaluating Deep Graph Generative Models

TutorialsDGX agent

arXiv:2609.01015v1 Announce Type: cross Abstract: Traditional network models from network science, such as the Erdos-Renyi and configuration models, generate random networks that reproduce few selecte

CompanionSim: Synthetic Data for Evaluating Anthropomorphism in Human-AI Relationships

Model ReleasesDGX agent

arXiv:2609.00250v1 Announce Type: cross Abstract: Many people now see AI systems as not just productivity tools but as social companions. Researchers are eager to study the consequences of AI companio

VIBE-Bench: Evaluating Personalized Large Language Models When Profiles Don't Mean Preferences

Model ReleasesDGX agent

arXiv:2609.00921v1 Announce Type: new Abstract: Personalized Large Language Models (PLLMs) aim to tailor responses to individual users, where a central challenge is preference reasoning: inferring que

1 Sept 2026

Evaluating Perspectival Biases in Cross-Modal Retrieval

Model ReleasesDGX agent

arXiv:2510.26861v4 Announce Type: replace-cross Abstract: Multimodal retrieval systems are expected to operate in a semantic space, agnostic to the language or cultural origin of the query. In practic

Evaluating the Hidden Costs of Personalization in Large Language Models

SafetyDGX agent

arXiv:2608.28833v1 Announce Type: new Abstract: While Large language models (LLMs) incorporate user personalization signals to improve usability and helpfulness, they increasingly shift from providing

Evaluating Tiny Recursive Models Across Training for Code Generation

Model ReleasesDGX agent

arXiv:2608.29376v1 Announce Type: new Abstract: Code generation increasingly relies on large transformer models, whose capability advances with scale. Yet such a scale is costly, creating demand for s

JFTA-Bench: Evaluate LLM's Ability of Tracking and Analyzing Malfunctions Using Fault Trees

Model ReleasesDGX agent

arXiv:2603.22978v2 Announce Type: replace Abstract: In the maintenance of complex systems, fault trees are used to locate problems and provide targeted solutions. To enable fault trees stored as image

JudgePanel: A Compact Judge with Panel Deliberation via Adaptive Multi-Reward Reinforcement Learning

AgentsDGX agent

arXiv:2608.29168v1 Announce Type: new Abstract: The LLM-as-a-Judge paradigm has emerged as a scalable alternative to human evaluation. However, single-model judges are limited by their inherent model

31 Aug 2026

Hypothesize, Evaluate, Refine: A Scientific Agent for PDE Discovery with Unknown Spatial Coefficient Fields

AgentsDGX agent

arXiv:2608.27475v1 Announce Type: cross Abstract: Discovering PDEs in heterogeneous media requires jointly identifying the governing operator and the unknown spatial fields that parameterize it. These

The Telephone Game: Evaluating Semantic Drift in Unified Models

Model ReleasesDGX agent

arXiv:2509.04438v3 Announce Type: replace-cross Abstract: Unified models (UMs) combine visual understanding (I2T) and generation (T2I) in a single framework. We focus on T2I and I2T, where cross-consi

28 Aug 2026

Evaluating Confidence-Gated Retrieval with Matched Trajectory Replay

Model ReleasesDGX agent

arXiv:2608.26846v1 Announce Type: cross Abstract: Interactive language-model agents use confidence signals to decide whether to answer immediately, retrieve additional evidence (from memory or externa

Evaluating human and LLM screening workflows in a conceptually complex scoping review: Recall--workload trade-offs and run-to-run consistency

Model ReleasesDGX agent

arXiv:2608.26885v1 Announce Type: new Abstract: Background. Large language models (LLMs) are increasingly used for screening in evidence synthesis, where false negatives can remove relevant studies be

R2M-Bench: Evaluating Revisit Memory via Relative Consistency in Interactive Video World Models

Model ReleasesDGX agent

arXiv:2608.27328v1 Announce Type: new Abstract: High similarity between first-visit and return frames does not necessarily show that a video world model remembered the scene; the intervening rollout m

26 Aug 2026

Evaluating Language Models on Cross-Language Code Functional Equivalence

Model ReleasesDGX agent

arXiv:2608.23961v1 Announce Type: cross Abstract: Background: Large Language Models (LLMs) have demonstrated strong performance across a variety of code-understanding tasks, leading many to believe th

Screening Autism Spectrum Disorder in children using Deep Learning Approach : Evaluating the classification model of YOLOv26s by comparing with other models

ResearchDGX agent

arXiv:2306.14300v2 Announce Type: replace-cross Abstract: Autism spectrum disorder (ASD) is a developmental condition that presents significant challenges in social interac- tion, communication, and b

25 Aug 2026

LongWoF-Bench: Evaluating EvoMap Genes for Verifiable Long-Workflow Tasks

Model ReleasesDGX agent

arXiv:2608.23200v1 Announce Type: new Abstract: Large language models are increasingly expected to execute complex workflows whose success depends on maintaining interdependent constraints and produci

MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesis

Model ReleasesDGX agent

arXiv:2608.22323v1 Announce Type: new Abstract: The application of Large Language Models (LLMs) to diagnostic decision-making has garnered growing interest. However, existing benchmarks largely focus

StrategyBench: Evaluating Explicit Strategy Induction in Large Language Models

Model ReleasesDGX agent

arXiv:2608.23475v1 Announce Type: new Abstract: As large language models are increasingly used in data-scarce and evolving task scenarios, few-shot in-context learning (ICL) has become a key paradigm

24 Aug 2026

CogCanvas: A Benchmark for Evaluating Multi-Subject Reference-Based Image Generation

Model ReleasesDGX agent

arXiv:2606.15867v2 Announce Type: replace Abstract: Multi-subject reference-based image generation requires jointly preserving multiple human identities, binding per-person objects and fashion items,

21 Aug 2026

Hallucination as a Feature, not a Defect: Evaluating a multi-agent architecture to transform speculative language-model outputs into testable scientific hypotheses

SafetyDGX agent

arXiv:2608.19206v1 Announce Type: cross Abstract: Contemporary Large Language Models (LLMs) are increasingly aligned to suppress hallucinations, prioritizing factual retrieval over combinatorial creat

Holtercare-Bench: A Multimodal Benchmark for Evaluating Long-Term Dynamic ECG Analysis

Model ReleasesDGX agent

arXiv:2608.19297v1 Announce Type: new Abstract: While multimodal large language models (MLLMs) excel in medical applications, most of them favor static images or short-term signals. In the critical fi

InsufficiencyBench: Evaluating LLM legal advice on underspecified user queries

Model ReleasesDGX agent

arXiv:2608.20220v1 Announce Type: new Abstract: Legal AI systems are increasingly used to answer legal questions, yet existing benchmarks assume queries arrive fully specified. In practice, users omit

20 Aug 2026

Metrics That Write Themselves: Evolving an Evaluator from Its Own Blind Spots

Model ReleasesDGX agent

arXiv:2608.18744v1 Announce Type: new Abstract: Agents improve quickly against a reliable automatic metric and stall without one, and the applications that need them most, report generation among them

SkillNet: Create, Evaluate, and Connect AI Skills

SafetyDGX agent

arXiv:2603.04448v2 Announce Type: replace Abstract: Current AI agents can flexibly invoke tools and execute complex tasks, yet their long-term advancement is hindered by the lack of systematic accumul

When Do LLMs Actually Help? Evaluating LLMs as Data Quality Annotators

Model ReleasesDGX agent

arXiv:2608.18158v1 Announce Type: cross Abstract: LLMs have been increasingly used to catch data quality issues automatically, but we know very little about how consistent these judgments actually are

18 Aug 2026

FabriMAE I Trust Myself? Self-Evaluating VLA Action Generation with Markov Attention Entropy

Model ReleasesDGX agent

arXiv:2608.16697v1 Announce Type: new Abstract: Vision-Language-Action models (VLAs) integrate visual perception, language instruction, and action generation into end-to-end policies across heterogene

FMReward: Aligning and Evaluating Audio-Driven 3D Facial Animation with Human Preferences

SafetyDGX agent

arXiv:2608.15296v1 Announce Type: new Abstract: Audio-driven 3D facial animation is essential for advancing immersion and interactivity in virtual experiences. Although recent advances have shown prom

Training and Evaluating Ethical Reinforcement Learning Agents on Per-Episode Distributions

Model ReleasesDGX agent

arXiv:2608.14642v1 Announce Type: new Abstract: Reinforcement Learning (RL) agents trained on a single reward signal exploit the gap between the designed reward and the intended behavior. This is part

14 Aug 2026

Beyond Simulated Benchmarks: Evaluating Motion Representations for Fall Detection Under Real-World Data Scarcity

ApplicationsDGX agent

arXiv:2608.13197v1 Announce Type: new Abstract: Falls are a major health concern for older adults, and wearable sensors have been widely explored for detecting falls and enabling timely intervention.

13 Aug 2026

CTBench: Evaluating Troubleshooting Capabilities of AI Agents in Realistic Telecom Network Operations

Model ReleasesDGX agent

arXiv:2608.12002v1 Announce Type: new Abstract: Agents are increasingly considered for automating network operations and maintenance, where engineers must diagnose network faults, optimize configurati

InfraBench: Evaluating Infrastructure Agents Across Layers, Lifecycle, and Risk

Model ReleasesDGX agent

arXiv:2608.11234v1 Announce Type: new Abstract: Managing modern computing infrastructure has become a steadily harder problem due to the ever-increasing complexity. Recent advances in AI agents create

TangPoetryBench: A Multi-Dimensional Benchmark and Rubric-Conditioned Evaluator for Poetry-to-Image Generation

Model ReleasesDGX agent

arXiv:2608.11452v1 Announce Type: cross Abstract: Text-to-image (T2I) models are increasingly asked to illustrate literary and cultural content, yet we cannot measure how well an image renders the mea

11 Aug 2026

Evaluating Generative Time-Series Models on Data with Point Masses

Model ReleasesDGX agent

arXiv:2608.09692v1 Announce Type: cross Abstract: Many of the series that generative time-series models are benchmarked on place a large probability mass on a single value --- it does not rain, no rid

How People Evaluate AI-, Expert-, and Peer-Style Financial Advice

ResearchDGX agent

arXiv:2608.09019v1 Announce Type: cross Abstract: As generative AI increasingly becomes a common source of daily decision-making, including financial choices, it is critical to understand how people e

OmnilingualGAIA2: Evaluating the Multilingual Gap in Frontier AI Agents

Model ReleasesDGX agent

arXiv:2608.08775v1 Announce Type: new Abstract: Agentic benchmarks aim to measure how well AI agents plan, search, execute, and recover within realistic multi-tool environments, but they are almost ex

TREAT: Evaluating Access to Formal Knowledge across Equivalent Mathematical Representations

Model ReleasesDGX agent

arXiv:2608.07540v1 Announce Type: new Abstract: AI systems increasingly operate between flexible input representations and formal objects used by downstream tools. A key challenge is recognizing when

WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Tasks

Model ReleasesDGX agent

arXiv:2506.01952v2 Announce Type: replace-cross Abstract: Powered by large language models (LLMs), web browsing agents operate graphical user interfaces in a human-like manner, offering a transparent

10 Aug 2026

Confirming Our Biases? Evaluating the Capabilities, Risks, and Societal Impact of Large Language Models

ResearchDGX agent

arXiv:2608.06977v1 Announce Type: new Abstract: It is well established that large language models (LLMs) are sensitive to prompt framing, reflecting patterns in their training data or prior prompts. I

HarnessSafe: Evaluating Safety Across Persistent Carriers in Agent Harnesses

Model ReleasesDGX agent

arXiv:2608.06984v1 Announce Type: cross Abstract: Modern agent harnesses persist state across tasks and sessions through persistent carriers like memory, skills, tools, and shared artifacts. However,

7 Aug 2026

CREBench: Evaluating Large Language Models in Cryptographic Binary Reverse Engineering

Model ReleasesDGX agent

arXiv:2604.03750v2 Announce Type: replace-cross Abstract: Reverse engineering (RE) is central to software security, particularly for cryptographic programs that handle sensitive data and are highly pr

EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents

Model ReleasesDGX agent

arXiv:2608.05519v1 Announce Type: new Abstract: Agent benchmarks usually measure task completion and treat resource use as an auxiliary statistic. In deployment, however, the choice among a local look

Evaluating Investment Logic in Large Language Models: A Real-World Benchmark Towards Personalzied Financial Agents

Model ReleasesDGX agent

arXiv:2608.06108v1 Announce Type: new Abstract: Investment competence is inherently personalized: the same market evidence can justify different actions for investors with different goals, horizons, p

← Previous
1…5960616263…687
Next →