AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “evaluation”

GridTimelineEvolution
41,190 results
9 Jun 2026

TempoBench: Evaluating Temporal Causal Reasoning in Large Language Models

Model ReleasesDGX agent

arXiv:2510.27544v2 Announce Type: replace Abstract: Temporal reasoning involves understanding how systems evolve over time through input-driven state transitions. A key aspect is temporal causal reaso

8 Jun 2026

Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle

Model ReleasesDGX agent

arXiv:2606.07462v1 Announce Type: new Abstract: As foundation models advance and agent scaffolding becomes increasingly sophisticated, agents have demonstrated remarkable proficiency in complex, long-

On the importance of multiple training seeds for evaluating machine unlearning

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
TutorialsDGX agent

arXiv:2510.26714v5 Announce Type: replace-cross Abstract: Machine unlearning aims to remove the influence of certain data points from a trained model without costly retraining. Most practical unlearni

Sycophantic Praise: Evaluating Excessive Praise in Language Models

SafetyDGX agent

arXiv:2606.07441v1 Announce Type: new Abstract: Sycophancy in language models is typically studied as excessive agreement or validation, while explicit praise and flattery have received comparatively

The Fine-Tuning Trap: Evaluating Negative Transfer and the Role of PEFT in Sub-1B Mathematical Reasoning

Model ReleasesDGX agent

arXiv:2606.06920v1 Announce Type: cross Abstract: Deploying Small Language Models (SLMs) on edge devices requires efficient fine-tuning strategies that adapt models to new tasks without degrading thei

6 Jun 2026

Evaluating Agentic Configuration Repair for Computer Networks

Model ReleasesDGX agent

arXiv:2606.06212v1 Announce Type: new Abstract: Misconfigurations in computer networks remain a major source of critical Internet outages. Research is turning to Large Language Models (LLMs) to automa

5 Jun 2026

CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives

Model ReleasesDGX agent

arXiv:2504.10823v4 Announce Type: replace Abstract: Navigating dilemmas involving conflicting values is challenging even for humans in high-stakes domains, let alone for AI, yet prior work has been li

4 Jun 2026

The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development?

Model ReleasesDGX agent

arXiv:2606.04455v1 Announce Type: new Abstract: Current AI benchmarks evaluate agents on task execution within human-designed workflows. These evaluations fundamentally fail to measure a critical next

'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions

Model ReleasesDGX agent

arXiv:2606.04906v1 Announce Type: cross Abstract: Although it is generally agreed that AI-generated text poses a broad societal risk, there is no common understanding in the AI-generated text detectio

3 Jun 2026

Evaluating and Calibrating LLM Confidence on Questions with Multiple Correct Answers

Model ReleasesDGX agent

arXiv:2602.07842v2 Announce Type: replace Abstract: Confidence calibration is essential for making large language models (LLMs) reliable, yet existing training-free methods have been primarily studied

Lingo_Research_Group at SemEval-2026 Task 9: Evaluating Prompt Variants for Polarization Detection

ResearchDGX agent

arXiv:2606.03334v1 Announce Type: new Abstract: Our submission presented in this paper is for SemEval-2026 Task 9: Multilingual Text Classification Challenge - Polarization Detection and it covers all

2 Jun 2026

Evaluating the Performance of Deep Learning Models in Whole-body Dynamic 3D Posture Prediction During Load-reaching Activities

ResearchDGX agent

arXiv:2511.20615v2 Announce Type: replace-cross Abstract: This study aimed to explore the application of deep neural networks for whole-body human posture prediction during dynamic load-reaching activ

KnowledgeBerg: Evaluating Systematic Knowledge Coverage and Compositional Reasoning in Large Language Models

Model ReleasesDGX agent

arXiv:2604.17621v2 Announce Type: replace Abstract: Many real-world questions appear deceptively simple yet implicitly demand two capabilities: (i) systematic coverage of a bounded knowledge universe

29 May 2026

BenchTrace: A Benchmark for Testing Reflection Ability and Controlled Evolution in LLM Agents

Model ReleasesDGX agent

arXiv:2605.29225v1 Announce Type: new Abstract: Self-evolving agents improve over time by reflecting on past failures, but existing evaluation is limited in two ways: it measures only task scores, lea

WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction

AgentsDGX agent

arXiv:2605.29341v1 Announce Type: cross Abstract: Multimodal large language models are increasingly deployed as long-horizon agents, where memory must do more than recall: it must track an evolving wo

27 May 2026

EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy

Model ReleasesDGX agent

arXiv:2605.24456v2 Announce Type: replace Abstract: Humans constantly reason about 3D proximity, the relations between their body and surrounding objects, to guide perception and action in daily life.

Real Images, Worse Judgments: Evaluating Vision-Language Models on Concreteness and Imagery

SafetyDGX agent

arXiv:2605.27315v1 Announce Type: new Abstract: Visual inputs are often assumed to improve language understanding in multimodal models. We examine this assumption by asking whether vision-language mod

25 May 2026

Evaluating Large Language Models in a Complex Hidden Role Game

Model ReleasesDGX agent

arXiv:2605.22826v1 Announce Type: cross Abstract: Quantifying the deceptive potential of Large Language Models (LLMs) is critical for AI safety, yet difficult to achieve in uncontrolled environments.

Evaluating Memory Structure in LLM Agents

Model ReleasesDGX agent

arXiv:2602.11243v2 Announce Type: replace-cross Abstract: Modern LLM-based agents and chat assistants rely on long-term memory frameworks to store reusable knowledge, recall user preferences, and augm

20 May 2026

Prediction Is Not Physics: Learning and Evaluating Conserved Quantities in Neural Simulators

SafetyDGX agent

arXiv:2605.18883v1 Announce Type: cross Abstract: A diffusion model trained on Hamiltonian trajectories can achieve rollout MSE near 10^{-3}, but the standard deviation of its energy over time is betw

19 May 2026

From Demographics to Survey Anchors: Evaluating LLM Agents for Modeling Retirement Attitudes

SafetyDGX agent

arXiv:2605.16303v1 Announce Type: cross Abstract: Large language models (LLM) agents may offer tools to predict human responses to surveys. A common technique for defining these agents uses only demog

18 May 2026

OgBench: A Framework for Evaluating Graph Neural Networks on Omics Data

Model ReleasesDGX agent

arXiv:2605.15511v1 Announce Type: new Abstract: Graph Neural Networks (GNNs) have become the dominant framework for inductive graph-level learning. Yet most benchmarks focus on the regime n gg p, wher

15 May 2026

Teaching and Evaluating LLMs to Reason About Polymer Design Related Tasks

Model ReleasesDGX agent

arXiv:2601.16312v2 Announce Type: replace-cross Abstract: Research in AI4Science has shown promise in many science applications, including polymer design. However, current LLMs are ineffective in this

14 May 2026

When Absolute State Fails: Evaluating Proprioceptive Encodings for Robust Manipulation

ApplicationsDGX agent

arXiv:2605.13067v1 Announce Type: cross Abstract: As end-to-end robotic policies are progressively deployed in the real world to solve real tasks, they face a gap between the training and inference co

12 May 2026

Active Testing of Large Language Models via Approximate Neyman Allocation

ResearchDGX agent

arXiv:2605.10075v1 Announce Type: new Abstract: Large language models (LLMs) require reliable evaluation from pre-training to test-time scaling, making evaluation a recurring rather than one-off cost.

11 May 2026

The Effective Depth Paradox: Evaluating the Relationship between Architectural Topology and Trainability in Deep CNNs

ResearchDGX agent

arXiv:2602.13298v3 Announce Type: replace-cross Abstract: This paper investigates the relationship between convolutional neural network (CNN) topology and image recognition performance through a compa

6 May 2026

Delay, Plateau, or Collapse: Evaluating the Impact of Systematic Verification Error on RLVR

ApplicationsDGX agent

arXiv:2605.02909v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a powerful approach for improving the reasoning capabilities of large language models (

4 May 2026

How Well Does GPT-4o Understand Vision? Evaluating Multimodal Foundation Models on Standard Computer Vision Tasks

Model ReleasesDGX agent

arXiv:2507.01955v3 Announce Type: replace Abstract: Multimodal foundation models (MFMs), such as GPT-4o, have recently made remarkable progress. However, their detailed visual understanding beyond que

30 Apr 2026

Evaluating the relationship between regularity and learnability in recursive numeral systems using Reinforcement Learning

TutorialsDGX agent

arXiv:2602.21720v2 Announce Type: replace-cross Abstract: Human recursive numeral systems (i.e., counting systems such as English base-10 numerals), like many other grammatical systems, are highly reg

28 Apr 2026

MarketBench: Evaluating AI Agents as Market Participants

Model ReleasesDGX agent

arXiv:2604.23897v1 Announce Type: new Abstract: Markets are a promising way to coordinate AI agent activity for similar reasons to those used to justify markets more broadly. In order to effectively p

Patterns vs. Patients: Evaluating LLMs against Mental Health Professionals on Personality Disorder Diagnosis through First-Person Narratives

Model ReleasesDGX agent

arXiv:2512.20298v2 Announce Type: replace-cross Abstract: Growing reliance on LLMs for psychiatric self-assessment raises questions about their ability to interpret qualitative patient narratives. Thi

27 Apr 2026

Evaluating LLM-Based Goal Extraction in Requirements Engineering: Prompting Strategies and Their Limitations

ResearchDGX agent

arXiv:2604.22207v1 Announce Type: cross Abstract: Due to the textual and repetitive nature of many Requirements Engineering (RE) artefacts, Large Language Models (LLMs) have proven useful to automate

24 Apr 2026

GeCo: Evaluating Geometric Consistency for Video Generation via Motion and Structure

Model ReleasesDGX agent

arXiv:2512.22274v2 Announce Type: replace Abstract: We introduce GeCo, a geometry-grounded metric for jointly detecting geometric deformation and occlusion-inconsistency artifacts in static scenes. By

23 Apr 2026

Evaluating Assurance Cases as Text-Attributed Graphs for Structure and Provenance Analysis

SafetyDGX agent

arXiv:2604.20577v1 Announce Type: cross Abstract: An assurance case is a structured argument document that justifies claims about a system's requirements or properties, which are supported by evidence

22 Apr 2026

Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language Models

SafetyDGX agent

arXiv:2511.06168v3 Announce Type: replace Abstract: This paper primarily demonstrates a method to quantitatively assess the alignment between multi-step, structured reasoning in large language models

The Cost of Relaxation: Evaluating the Error in Convex Neural Network Verification

ResearchDGX agent

arXiv:2604.18728v1 Announce Type: cross Abstract: Many neural network (NN) verification systems represent the network's input-output relation as a constraint program. Sound and complete, representatio

21 Apr 2026

Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users

Model ReleasesDGX agent

arXiv:2603.16120v2 Announce Type: replace Abstract: Deep Research (DR) systems help researchers cope with ballooning publishing counts. Such tools synthesize scientific papers to answer research queri

17 Apr 2026

The Acoustic Camouflage Phenomenon: Re-evaluating Speech Features for Financial Risk Prediction

ResearchDGX agent

arXiv:2604.14619v1 Announce Type: cross Abstract: In computational paralinguistics, detecting cognitive load and deception from speech signals is a heavily researched domain. Recent efforts have attem

14 Apr 2026

Evaluating Cooperation in LLM Social Groups through Elected Leadership

AgentsDGX agent

arXiv:2604.11721v1 Announce Type: cross Abstract: Governing common-pool resources requires agents to develop enduring strategies through cooperation and self-governance to avoid collective failure. Wh

MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion

SafetyDGX agent

arXiv:2604.09587v1 Announce Type: new Abstract: Mobile agents can autonomously complete user-assigned tasks through GUI interactions. However, existing mainstream evaluation benchmarks, such as Androi

13 Apr 2026

Act or Escalate? Evaluating Escalation Behavior in Automation with Language Models

SafetyDGX agent

arXiv:2604.08588v1 Announce Type: cross Abstract: Effective automation hinges on deciding when to act and when to escalate. We model this as a decision under uncertainty: an LLM forms a prediction, es

10 Apr 2026

Evaluating PQC KEMs, Combiners, and Cascade Encryption via Adaptive IND-CPA Testing Using Deep Learning

ResearchDGX agent

arXiv:2604.06942v1 Announce Type: cross Abstract: Ensuring ciphertext indistinguishability is fundamental to cryptographic security, but empirically validating this property in real implementations an

FMI@SU ToxHabits: Evaluating LLMs Performance on Toxic Habit Extraction in Spanish Clinical Texts

Model ReleasesDGX agent

arXiv:2604.06403v1 Announce Type: cross Abstract: The paper presents an approach for the recognition of toxic habits named entities in Spanish clinical texts. The approach was developed for the ToxHab

VAREX: A Benchmark for Multi-Modal Structured Extraction from Documents

Model ReleasesDGX agent

arXiv:2603.15118v2 Announce Type: replace Abstract: We introduce VAREX (VARied-schema EXtraction), a benchmark for evaluating multimodal foundation models on structured data extraction from government

15 Oct 2026

A Node Embedding Framework for Model-Centric Conformance Checking in Process Mining

Local AiDGX agent

arXiv:2610.17702v1 Announce Type: new Abstract: Evaluating the quality of process models discovered from event logs is a fundamental challenge in process mining. Traditional evaluation relies on confo

Benchmarking and Training Reward Models for Cross-Source Evidence Grounding in Search Agents

Model ReleasesDGX agent

arXiv:2610.17827v1 Announce Type: new Abstract: Existing approaches to training and evaluating search agents focus heavily on final-answer quality, largely neglecting how generated answers map back to

In-Context Batch Black-box Optimization via Flow Matching

ApplicationsDGX agent

arXiv:2610.20138v1 Announce Type: new Abstract: Batch black-box optimization is important for expensive systems where many candidates can be evaluated in parallel, yet only a few rounds of evaluation

LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering

Model ReleasesDGX agent

arXiv:2608.28281v2 Announce Type: replace Abstract: Loop Engineering is emerging as a practice for organizing development work around coding agents. Instead of writing each prompt by hand, practitione

PhyMetric: Diagnosing Physical Plausibility in Text-to-Video Generation via Scene-Level QA

Model ReleasesDGX agent

arXiv:2610.19823v1 Announce Type: new Abstract: Text-to-video (T2V) generation models can produce visually realistic videos that nevertheless violate the physical consequences implied by the prompt an

PMO-Dock: Benchmarking Docking, Specificity, and Generalization in Molecular Optimization

Model ReleasesDGX agent

arXiv:2610.19011v1 Announce Type: new Abstract: The Practical Molecular Optimization (PMO) benchmark standardized evaluation in molecular optimization, but it is built on simple property-based oracles

QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models

Model ReleasesDGX agent

arXiv:2601.08689v3 Announce Type: replace Abstract: Large Language Models (LLMs) have shown strong capabilities in finance-oriented question answering, yet their evaluation on professional quantitativ

14 Oct 2026

CAG-WAM: Causal Action-Gain World Action Models for Robot Manipulation

SafetyDGX agent

arXiv:2610.14720v1 Announce Type: new Abstract: World Action Models (WAMs) combine action generation with future prediction, allowing predicted futures to support action evaluation and robot decision

EmoTrace-Bench: Diagnosing Emotional Expression in Text-to-Video Generation through Controlled Cues and Intensity

Model ReleasesDGX agent

arXiv:2610.15452v1 Announce Type: new Abstract: Text-to-video (T2V) models are increasingly used to generate videos with targeted emotional expressions, yet existing evaluation methods mainly rely on

WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executions

Model ReleasesDGX agent

arXiv:2610.18826v1 Announce Type: cross Abstract: We introduce WorldOracle, a benchmark specifically designed to evaluate action-conditioned world models (ACWMs) against paired real-world executions.

9 Oct 2026

EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolution

AgentsDGX agent

arXiv:2610.12086v1 Announce Type: new Abstract: LLM-based program evolution relies on evaluation feedback to guide the iterative search for high-performing programs. However, evaluation is often compu

Obscuring Data Contamination Through Translation: Evidence from Arabic Corpora

Model ReleasesDGX agent

arXiv:2601.14994v2 Announce Type: replace-cross Abstract: Data contamination can invalidate benchmark evaluation when a model benefits from memorized evaluation content rather than genuine generalizat

5 Oct 2026

Reliability Stress Tests and Decision-Time Routing for Chest X-ray Vision-Language Models

Model ReleasesDGX agent

arXiv:2610.02270v1 Announce Type: cross Abstract: Medical vision-language model (VLM) evaluation is sensitive to workflow design, prompting strategy, and benchmark construction, yet most studies treat

2 Oct 2026

PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Models

Model ReleasesDGX agent

arXiv:2610.01162v1 Announce Type: new Abstract: Reliable video world models could provide scalable predictive environments for robot learning, planning, and evaluation. However, generated robot videos

1 Oct 2026

A2Z GameSpec-Bench: How Faithfully Can Coding Agents Generate Games from Game Design Specifications?

Model ReleasesDGX agent

arXiv:2609.39564v1 Announce Type: new Abstract: Delegating complete application development to coding agents requires preserving the intended design rather than simply producing plausible outputs thro

30 Sept 2026

'Shall We Play a Game?' - H3 - Wargames(83) ending [Spoiler] recreated using 4 stills and a sample of Joshua's voice. (Claude Fable on 3rd try vs. Astra)

Model ReleasesDGX agent

Generated with H3: Mock H3 version of the ending to Wargames(83) as a test of a custom node I've been working on w/ both Claude(Opus/Fable) and OAI's Sol/Astra. One click generation using only 4 movie

← Previous
1…7071727374…687
Next →