AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “evaluation”

GridTimelineEvolution
41,190 results
17 Apr 2026

Anomaly Detection in IEC-61850 GOOSE Networks: Evaluating Unsupervised and Temporal Learning for Real-Time Intrusion Detection

ResearchDGX agent

arXiv:2604.14233v1 Announce Type: cross Abstract: The IEC-61850 GOOSE protocol underpins time-critical communication in modern digital substations but lacks native security mechanisms, leaving it vuln

Knowing When Not to Answer: Evaluating Abstention in Multimodal Reasoning Systems

Model ReleasesDGX agent

arXiv:2604.14799v1 Announce Type: new Abstract: Effective abstention (EA), recognizing evidence insufficiency and refraining from answering, is critical for reliable multimodal systems. Yet existing e

QuantCode-Bench: A Benchmark for Evaluating the Ability of Large Language Models to Generate Executable Algorithmic Trading Strategies

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Model Releases
DGX agent

arXiv:2604.15151v1 Announce Type: new Abstract: Large language models have demonstrated strong performance on general-purpose programming tasks, yet their ability to generate executable algorithmic tr

16 Apr 2026

MulDimIF: A Multi-Dimensional Constraint Framework for Evaluating and Improving Instruction Following in Large Language Models

Model ReleasesDGX agent

arXiv:2505.07591v2 Announce Type: replace Abstract: Instruction following refers to the ability of large language models (LLMs) to generate outputs that satisfy all specified constraints. Existing res

15 Apr 2026

AgenticAI-DialogGen: Topic-Guided Conversation Generation for Fine-Tuning and Evaluating Short- and Long-Term Memories of LLMs

AgentsDGX agent

arXiv:2604.12179v1 Announce Type: new Abstract: Recent advancements in Large Language Models (LLMs) have improved their ability to process extended conversational contexts, yet fine-tuning and evaluat

Evaluating LLM-Generated ACSL Annotations for Formal Verification

Model ReleasesDGX agent

arXiv:2602.13851v3 Announce Type: replace-cross Abstract: Formal specifications are crucial for building verifiable and dependable software systems, yet generating accurate and verifiable specificatio

14 Apr 2026

ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents

AgentsDGX agent

arXiv:2604.11784v1 Announce Type: cross Abstract: GUI agents drive applications through their visual interfaces instead of programmatic APIs, interacting with arbitrary software via taps, swipes, and

CounterBench: Evaluating and Improving Counterfactual Reasoning in Large Language Models

Model ReleasesDGX agent

arXiv:2502.11008v2 Announce Type: replace Abstract: Counterfactual reasoning is widely recognized as one of the most challenging and intricate aspects of causality in artificial intelligence. In this

Evaluating Memory Capability in Continuous Lifelog Scenario

Model ReleasesDGX agent

arXiv:2604.11182v1 Announce Type: new Abstract: Nowadays, wearable devices can continuously lifelog ambient conversations, creating substantial opportunities for memory systems. However, existing benc

Evaluating the Impact of Medical Image Reconstruction on Downstream AI Fairness and Performance

SafetyDGX agent

arXiv:2604.10904v1 Announce Type: cross Abstract: AI-based image reconstruction models are increasingly deployed in clinical workflows to improve image quality from noisy data, such as low-dose X-rays

If an LLM Were a Character, Would It Know Its Own Story? Evaluating Lifelong Learning in LLMs

Model ReleasesDGX agent

arXiv:2503.23514v2 Announce Type: replace-cross Abstract: Large language models (LLMs) can carry out human-like dialogue, but unlike humans, they are stateless due to the superposition property. Howev

SecureVibeBench: Evaluating Secure Coding Capabilities of Code Agents with Realistic Vulnerability Scenarios

Model ReleasesDGX agent

arXiv:2509.22097v3 Announce Type: replace-cross Abstract: Large language model-powered code agents are rapidly transforming software engineering, yet the security risks of their generated code have be

13 Apr 2026

SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessment

Model ReleasesDGX agent

arXiv:2604.08988v1 Announce Type: new Abstract: Current LLM-based agents demonstrate strong performance in episodic task execution but remain constrained by static toolsets and episodic amnesia, faili

10 Apr 2026

Evaluating In-Context Translation with Synchronous Context-Free Grammar Transduction

ResearchDGX agent

arXiv:2604.07320v1 Announce Type: cross Abstract: Low-resource languages pose a challenge for machine translation with large language models (LLMs), which require large amounts of training data. One p

ModeX: Evaluator-Free Best-of-N Selection for Open-Ended Generation

Model ReleasesDGX agent

arXiv:2601.02535v2 Announce Type: replace Abstract: Selecting a single high-quality output from multiple stochastic generations remains a fundamental challenge for large language models (LLMs), partic

16 Oct 2026

Abstracts or full text? Evaluating input scope and computational efficiency in biomedical lay summarization

ResearchDGX agent

arXiv:2610.21690v1 Announce Type: new Abstract: Biomedical research articles are mainly written for experts and often contain terms that are difficult for the general public to understand. Automated l

DialFork: Going Beyond Factuality to Evaluate Procedural Hallucination in Conversational Agents

Model ReleasesDGX agent

arXiv:2610.21561v1 Announce Type: cross Abstract: Large language models (LLMs) deployed as conversational agents are increasingly expected to follow procedural logic. While factual hallucination has r

QuasiLeak-Bench: Evaluating LLM-Based Privacy Detectors on Policy-Conditioned Quasi-Evidence Group Recovery

Model ReleasesDGX agent

arXiv:2610.21333v1 Announce Type: new Abstract: Natural language messages often contain quasi-identifiers that are weak in isolation but can jointly reveal an individual's private attributes. We call

15 Oct 2026

SWEPatchTest: Evaluating Software Tests Beyond Reproduction

Model ReleasesDGX agent

arXiv:2610.18004v1 Announce Type: cross Abstract: Coding agents use generated tests to decide whether a proposed patch satisfies an issue's requirements. A suite that fails on the buggy program and pa

14 Oct 2026

Not All Challenges Are Equal: Evaluating Interactive Liveness Detection Against Real-Time Face Swaps

ResearchDGX agent

arXiv:2610.16417v1 Announce Type: new Abstract: Remote identity verification assumes that a camera stream shows a live person, yet public real-time face-swap tools can replace that person before the s

12 Oct 2026

Evaluating LLM Creativity through Long-Tail Performance

Model ReleasesDGX agent

arXiv:2610.12998v1 Announce Type: new Abstract: Creativity benchmarks for large language models (LLMs) vary widely in task and metric design. This raises the question of whether they capture a common

KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse

ResearchDGX agent

arXiv:2610.13060v1 Announce Type: new Abstract: KV-cache reuse in large language model serving is commonly limited to exact prompt prefixes, preventing reuse when the same retrieved document appears a

QTestBench: A Test Framework for Evaluating LLM-Generated Quantum Code

Model ReleasesDGX agent

arXiv:2610.13811v1 Announce Type: cross Abstract: Large language models (LLMs) have shown strong results in classical code generation and are increasingly used to write the unit tests that safeguard t

TabReliBench: Evaluating Tabular Foundation Models Beyond Predictive Accuracy

Model ReleasesDGX agent

arXiv:2610.13654v1 Announce Type: new Abstract: Tabular foundation models (TFMs) are increasingly popular for their strong predictive performance. Many also provide posterior predictive distributions

What Should Embodied Agents Remember? Benchmarking and Evaluating Memory Modality for Embodied Agents

Model ReleasesDGX agent

arXiv:2610.12561v1 Announce Type: new Abstract: Memory allows large language model (LLM) agents to use past experience, often stored as textual records, when responding to requests. For embodied agent

9 Oct 2026

Can Decision Models Understand Stance? Evaluating Jev Against General-Purpose LLMs

Model ReleasesDGX agent

arXiv:2610.11901v1 Announce Type: new Abstract: Stance detection requires identifying an author's attitude toward a given target, sometimes based on conversational context. Jev, a specialized decision

Evaluating Exact Output and Checkpoint-State Prediction in Real Programs

Model ReleasesDGX agent

arXiv:2610.11889v1 Announce Type: cross Abstract: We present a benchmark for predicting final output and checkpoint state from source and input alone. It extends CRUXEval-style output prediction with

WorldBench: Evaluating LLMs on Three.js Voxel World Generation

Model ReleasesDGX agent

arXiv:2610.10622v1 Announce Type: cross Abstract: Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges tak

8 Oct 2026

Evaluating Sequence Assembly Strategies for Differentially Private Synthetic Time-Series Forecasting

ResearchDGX agent

arXiv:2610.10222v1 Announce Type: new Abstract: Differentially private time-series generators commonly produce fixed-length synthetic windows, whereas downstream forecasting models often require long

From Plausible Hierarchies to Useful Taxonomies: Evaluating Agentic Harnesses on Customer Feedback

AgentsDGX agent

arXiv:2610.09377v1 Announce Type: cross Abstract: Taxonomies are the symbolic representations through which AI systems organize evidence, aggregate patterns, and answer questions over large document c

GAGR-Lab: Evaluating Joint Spatial-Geometric and Analytic Function Reasoning

Model ReleasesDGX agent

arXiv:2610.10201v1 Announce Type: cross Abstract: Joint spatial-geometric and analytic function reasoning requires translating a perceived spatial configuration into a symbolic function whose executed

Shared-Roadmap Generation and Evaluator for Multi-Agent Path Planning Using Heterogeneous Graph Neural Network

SafetyDGX agent

arXiv:2610.09034v1 Announce Type: cross Abstract: Multi-agent path planning (MAPP) in continuous environments often relies on roadmaps to balance safety and search efficiency. However, traditional roa

7 Oct 2026

Beyond Task Reward: A Controller-Restriction Protocol for Evaluating Embodiment-Dependent Competence

ResearchDGX agent

arXiv:2610.07629v1 Announce Type: new Abstract: Co-design methods optimize a robot's body and controller jointly and judge the result by one number, the task reward of the fully optimized pair. That n

Evaluating Escalation Signals for LLM Routing: Targets, Controls, and Five Ways to Fool Yourself

Model ReleasesDGX agent

arXiv:2610.07354v1 Announce Type: new Abstract: Deciding when to escalate a query from a small language model to a larger one requires a cheap signal that predicts, before the large model is called, w

Evaluating Inference Compute for Generative AI: A Framework for Enterprise Workloads

Model ReleasesDGX agent

arXiv:2610.07094v1 Announce Type: cross Abstract: LLM deployment is shifting from single-turn completion to agentic trajectories in which a model plans, calls tools, reads results and reasons at test

What pass@k Cannot Measure: Evaluating Diversity and Capability Retention after Post-Training

SafetyDGX agent

arXiv:2610.07405v1 Announce Type: new Abstract: pass@k, the fraction of problems a model solves within k sampled attempts, is the field's default protocol for deciding whether reinforcement-learning (

6 Oct 2026

Why Convolution Still Matters: Evaluating Inductive Biases in Cryospheric Image Classification

Model ReleasesDGX agent

arXiv:2610.04073v1 Announce Type: new Abstract: Recent advances in attention-based deep learning have motivated their adoption for remote sensing image classification; however, their benefits for cryo

5 Oct 2026

Ask, Relax, or Act? Evaluating Actionable Indeterminacy in LLM Preference Reasoning

Model ReleasesDGX agent

arXiv:2610.03102v1 Announce Type: cross Abstract: An LLM agent can recognize uncertainty yet still choose the wrong next step: asking when action is already justified, or seeking clarification when th

CriticHack: Evaluating Visual Rewards Under Robot Policy Optimization

Model ReleasesDGX agent

arXiv:2610.02527v1 Announce Type: cross Abstract: Learned visual reward models are increasingly used to optimize robot policies, yet a reward model can score an execution that acts on the wrong object

To Jev or Not? Evaluating the Accuracy and Efficiency of Structured Decision Models for Hate-Speech Moderation

ResearchDGX agent

arXiv:2610.03324v1 Announce Type: new Abstract: The scale of online content makes hate-speech moderation challenging, while Large Language Models (LLMs) enable harmful material to be produced and adap

2 Oct 2026

The COTe score: A decomposable framework for evaluating Document Layout Analysis models

ResearchDGX agent

arXiv:2603.12718v3 Announce Type: replace Abstract: Document Layout Analysis (DLA) is the process by which a page is parsed into meaningful elements, often using machine learning models. Typically, th

1 Oct 2026

Better Deck or Different Judge? Evaluating Agentic Harness Gains in Corporate and Investment Banking

AgentsDGX agent

arXiv:2609.39958v1 Announce Type: new Abstract: Corporate and investment banking teams use presentations to support credit decisions and advise clients on financing and transactions. Producing these d

DAGent: Evaluate-then-Grow Planning for Deep Research Agents

Model ReleasesDGX agent

arXiv:2609.39154v1 Announce Type: cross Abstract: Deep research tasks require agents to navigate large knowledge spaces, synthesize evidence across many sources, and adapt their plans as findings emer

FANVIDv2: Evaluating Video Super-Resolution by Face and Licence-Plate Recognition Under Compound Degradation

Model ReleasesDGX agent

arXiv:2609.39649v1 Announce Type: new Abstract: Video super-resolution (VSR) is normally judged by PSNR and SSIM on clips that were downsampled bicubically, although in surveillance its purpose is to

LIBERO-Agent: Evaluating General-Purpose Agents for Direct Embodied Manipulation

Model ReleasesDGX agent

arXiv:2609.39507v1 Announce Type: new Abstract: General-purpose agents can plan, use tools, and revise their behavior from feedback, but it remains unclear whether these capabilities transfer from dig

Who Owns That? Evaluating Ownership Intuitions in Large Language Models

ResearchDGX agent

arXiv:2609.39483v1 Announce Type: new Abstract: Ownership establishes rights over the use, control, and transfer of objects. Understanding these relations is essential for AI systems to interact appro

30 Sept 2026

Bits Under ZK-LLM: Evaluating Zero-Knowledge-Friendly Quantization for Verifiable Private LLM Inference

ResearchDGX agent

arXiv:2609.36437v1 Announce Type: new Abstract: Zero-knowledge proofs are emerging as a promising approach for enabling private, verifiable LLM governance and auditing, where regulators, users, and au

CIRCLE: A Framework for Evaluating AI from a Real-World Lens

ApplicationsDGX agent

arXiv:2602.24055v5 Announce Type: replace Abstract: This study proposes CIRCLE, a six-stage, lifecycle-based framework to bridge the reality gap between model-centric performance metrics and AI system

DatalogBench: Evaluating Large Language Models on Text-to-Datalog Synthesis

Model ReleasesDGX agent

arXiv:2609.37233v1 Announce Type: cross Abstract: Datalog underpins reasoning tasks such as program analysis, but its programs are hard to write. Existing synthesizers automate this task but require u

VIF-Bench: Evaluating Visual Instruction Following in Multi-Reference Image Generation

Model ReleasesDGX agent

arXiv:2609.37709v1 Announce Type: cross Abstract: Recent multimodal image generation models can take multiple images and textual instructions as input, enabling reference-based generation guided not o

What You Observe Determines How You Identify Causal Effects: Evaluating Causal Models across Observational Views

Model ReleasesDGX agent

arXiv:2609.36881v1 Announce Type: new Abstract: Causal foundation models (CFMs) pre-trained on data generated from various structural causal models (SCMs) have been proposed for estimating causal effe

When Tools Silently Lie: Evaluating and Mitigating Blind Compliance in Tool-Augmented Data Agents

AgentsDGX agent

arXiv:2609.37153v1 Announce Type: new Abstract: Tool-augmented data agents rely on tool outputs for analytical decisions. Yet successful execution can return plausible but incorrect evidence, requirin

29 Sept 2026

Can't Find Waldo: Evaluating VLMs' Sensitivity to Image Resolution and Detail Level

ResearchDGX agent

arXiv:2609.31706v1 Announce Type: new Abstract: Visual Language Models (VLMs) have achieved remarkable success across diverse tasks, yet they struggle with high-resolution inputs where critical inform

DynamicDx: Evaluating Evidence Acquisition in Video-Based Diagnosis

ResearchDGX agent

arXiv:2609.32957v1 Announce Type: new Abstract: Diagnosing a patient from video requires more than recognizing the sign: a vision-language model must turn what it sees into hypotheses, questions and t

Evaluating Hierarchy-Aware Deep Learning for the Recognition of Tironian Notes

ResearchDGX agent

arXiv:2609.35277v1 Announce Type: new Abstract: Tironian notes are generally regarded as the first Latin shorthand system and are notable for their large, fine-grained symbol inventory. Their high vis

GUI-GenBench: Evaluating Image Generation Models as Interactive GUI Environments

Model ReleasesDGX agent

arXiv:2602.09007v3 Announce Type: replace-cross Abstract: An image generation model used as a graphical user interface (GUI) environment must produce not only a plausible screen but also the correct r

Toward AI-Assisted Poultry Coccidiosis Diagnosis: Evaluating Gemini and BiomedParse on Eimeria Microscopy Images

Model ReleasesDGX agent

arXiv:2609.31679v1 Announce Type: new Abstract: Coccidiosis caused by Eimeria parasites is a major economic burden in poultry production, and effective control depends on accurate species-level diagno

28 Sept 2026

Autoheal raises $7.9M to evaluate and fix AI agents with… AI agents

IndustryDGX agent

Autoheal AI Inc., an artificial intelligence-native platform engineering startup that’s trying to pioneer the concept of “self-improving software factories,” said today it has raised 7.9 million in se

Evaluating the Impact of Adaptive Extended Reality on Human-Robot Interaction Across the Reality-Virtuality Continuum

ResearchDGX agent

arXiv:2609.31138v1 Announce Type: new Abstract: As populations in developed countries age and labor shortages intensify, Cybernetic Avatars (CAs) are proposed to extend human capabilities through robo

26 Sept 2026

My Reading Library: Evaluating LLMs on Android Tasks

Local AiDGX agent

Can LLM agents actually get through a day in the life of a normal user? That question got me reading papers on Android agents and mobile benchmarks over the past few months. A few patterns kept showin

← Previous
1…6364656667…687
Next →