AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “evaluation”

GridTimelineEvolution
41,190 results
25 Sept 2026

From Policy Documents to Structured Survey Responses: Evaluating Large Language Models for Policy Monitoring

SafetyDGX agent

arXiv:2609.29370v1 Announce Type: cross Abstract: Science, technology, and innovation policies are crucial for competitiveness, yet their diversity and scale make them difficult to map and monitor con

LabFactory: Building and Evaluating Executable AI Labs

AgentsDGX agent

arXiv:2609.28697v1 Announce Type: new Abstract: Scientific tasks specify a desired capability, but realizing it often requires building a computational system tailored to the task---acquiring data, de

WebArxiv: A Reproducible Benchmark for Evaluating Multimodal Web Agents on arXiv Tasks

Model ReleasesDGX agent

arXiv:2507.00938v4 Announce Type: replace-cross Abstract: Foundation models now enable autonomous agents to interact with real-world websites, but existing benchmarks emphasize general-purpose browsin

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
24 Sept 2026

Evaluating Feedback Focus and Pedagogical Adaptivity in LLM-Generated Feedback on Student Writing

Model ReleasesDGX agent

arXiv:2609.28026v1 Announce Type: cross Abstract: We investigate whether state-of-the-art large language models (LLMs) generate feedback that reflects the pedagogical practices of expert teachers in t

X2Real: an eXtensive simulation benchmark for real-world generalist policies

Model ReleasesDGX agent

arXiv:2609.27449v1 Announce Type: new Abstract: Generalist robot manipulation policies have developed rapidly, yet their reliable evaluation remains challenging due to fundamental flaws in existing si

23 Sept 2026

A Semiotics-Aware Framework for Evaluating Fidelity and Coverage in Natural Language Generation

SafetyDGX agent

arXiv:2609.26527v1 Announce Type: new Abstract: When two texts describe the same expression, standard metrics based on lexical overlap or whole-text similarity may fail to detect meaningful difference

BackTrend: Evaluating Scientific Weak-Signal Prediction via Backward Reconstruction

Model ReleasesDGX agent

arXiv:2609.24921v1 Announce Type: new Abstract: Scientific weak signals are early, low-visibility research directions that later become central to mature scientific topics, yet existing resources such

Evaluating Accuracy and Probabilistic Reliability of Zero-Shot Time Series Foundation Models

Model ReleasesDGX agent

arXiv:2609.25788v1 Announce Type: new Abstract: Time Series Foundation Models (TSFMs) promise a paradigm shift toward zero-shot forecasting by eliminating task-specific training. However, existing wor

Evaluating the Effectiveness of SechKAN on 1D Data

Model ReleasesDGX agent

arXiv:2609.25876v1 Announce Type: new Abstract: The connection between the Kolmogorov-Arnold representation theorem (KART) and neural network design has led to the development of Kolmogorov-Arnold Net

22 Sept 2026

A Comparative Framework for Evaluating Foundation Models on Tabular Data: A Case Study in Healthcare

ApplicationsDGX agent

arXiv:2609.22154v1 Announce Type: new Abstract: Tabular data is the most common format in clinical practice, encompassing laboratory results, medication records, diagnostic codes, and patient demograp

ChartJudgeBench: Evaluating LMM Judges for Chart-to-Code Generation

Model ReleasesDGX agent

arXiv:2609.24210v1 Announce Type: new Abstract: Building strong chart-to-code systems increasingly relies on reinforcement learning, whose effectiveness depends critically on the quality of the reward

Do LLMs Choose Like Humans? Using Cognitive Theory to Evaluate LLM Decision-Making

Model ReleasesDGX agent

arXiv:2609.22225v1 Announce Type: new Abstract: Large language models (LLMs) exhibit a range of human-like decision-making behaviors, but whether these reflect similar underlying mechanisms or surface

Evaluate skill-equipped agents with Strands Evals and Amazon Bedrock AgentCore

AgentsDGX agent

Skills let you encode domain-specific procedures as reusable, portable instructions for agents, but a fluent answer doesn't prove the agent picked the right skill or followed it. Learn how to measure

Evaluating the Generalization of Neuroimaging Foundation Models on African Brain MRI

Model ReleasesDGX agent

arXiv:2609.23983v1 Announce Type: new Abstract: Neuroimaging foundation models pretrained on large, predominantly western cohorts are increasingly proposed as general-purpose backbones for brain MRI a

MSI-Bench: Evaluating Multi-Speaker Voice Interaction for Collaborative AI Agents

Model ReleasesDGX agent

arXiv:2609.24812v1 Announce Type: new Abstract: Voice provides a natural and immediate interface for AI agents. Many settings in which voice agents could be useful, including meetings, households, and

To Consolidate or not to Consolidate? Evaluating the Impact of Consolidation in Multi-Reference Training using Peer Reviews

Model ReleasesDGX agent

arXiv:2609.22805v1 Announce Type: new Abstract: Natural language generation (NLG) tasks span the spectrum of conditional entropy, ranging from highly constrained machine translation to open-ended dial

21 Sept 2026

Evaluating In-Context Learning and Retrieval Strategies for Devanagari Post-OCR Correction

Model ReleasesDGX agent

arXiv:2609.21595v1 Announce Type: new Abstract: In-context learning using Large Language Models (LLMs) offers a compelling path to training-free post-OCR correction, yet its effectiveness for Devanaga

18 Sept 2026

Evaluating Financial Sentiment in the Age of AI

ApplicationsDGX agent

arXiv:2609.20198v1 Announce Type: new Abstract: Financial sentiment measures are widely used in empirical finance, but it remains unclear whether general-purpose large language models (LLMs) improve o

Evaluating Out-of-Distribution Robustness in Graph-Based Android Malware Classification: A New Principled Benchmark

Model ReleasesDGX agent

arXiv:2508.06734v3 Announce Type: replace-cross Abstract: While graph-based Android malware classifiers report strong benchmark accuracy of over 94%, their performance sharply decreases up to 45% when

17 Sept 2026

Prepared Or Unprepared? Evaluating Healthcare Workforce Readiness for Clinical Adoption of Artificial Intelligence in Nigeria

ApplicationsDGX agent

arXiv:2609.19096v1 Announce Type: cross Abstract: Artificial intelligence (AI) is increasingly integrated into healthcare systems worldwide, yet its successful clinical adoption depends critically on

16 Sept 2026

Can LLMs Follow the Pulse of a Crisis? Evaluating Crisis Sentiment in Bangladesh's July Uprising

Model ReleasesDGX agent

arXiv:2609.16997v1 Announce Type: new Abstract: Crisis sentiment analysis is especially challenging for low-resource languages such as Bangla, where language, context, and public reaction shift rapidl

Evaluating Mesh Reconstruction Methods for Crop Phenotyping

ApplicationsDGX agent

arXiv:2609.16926v1 Announce Type: new Abstract: Phenotyping an agricultural crop is crucial for studying its entire life cycle, as it provides vital insights to improve yield and, ultimately, food pro

LSREP: A Longitudinal State-Replay Protocol for Evaluating Conversational Memory, with ICE v2 as an Audited Local-First Architecture

ApplicationsDGX agent

arXiv:2609.16730v1 Announce Type: new Abstract: Conversational memory changes during use, so endpoint question answering alone cannot establish how a persistent state accumulates, ages, or incorporate

15 Sept 2026

CoArena: Evaluating Computer-Use and Multi-Agent Systems in Real Time

AgentsDGX agent

arXiv:2609.14239v1 Announce Type: new Abstract: Static benchmarks for computer-use agents fix a task set at release and score every system against it once. That makes them reproducible, and it lets th

RFCLLM: Evaluating LLMs' Reasoning Ability of Network Protocol State Machines

ResearchDGX agent

arXiv:2609.13389v1 Announce Type: new Abstract: Mapping textual specifications into formal representations is essential for ensuring the correctness of protocol designs and implementations. LLM-genera

14 Sept 2026

ParaRecover: A Process-Level Benchmark for Error Localization and Recovery in Parallel Tool-Use Agents

Model ReleasesDGX agent

arXiv:2609.12345v1 Announce Type: new Abstract: Existing agent benchmarks mainly evaluate final task success or tool-call correctness, providing limited insight into whether agents can reliably diagno

11 Sept 2026

PEARL: A Task-Aware Framework for Evaluating Differentially Private Synthetic Educational Data

SafetyDGX agent

arXiv:2609.10612v1 Announce Type: cross Abstract: Personalized learning systems rely on real learner data, including performance, behavior, and demographic information, but these data are highly priva

10 Sept 2026

DI-Bench: Systematically Generating In-Domain Data Intelligence Benchmarks for Enterprise Agents

Model ReleasesDGX agent

arXiv:2609.05776v1 Announce Type: new Abstract: Evaluating enterprise agents on domain-specific benchmarks is critical, yet public benchmarks rarely evaluate whether agents can integrate business know

Evaluating the Scalability and Adversarial Generalization of GRPO-Trained NLI Models

Model ReleasesDGX agent

arXiv:2504.18376v3 Announce Type: replace-cross Abstract: Natural Language Inference (NLI) is a central task in natural language understanding with applications in fact-checking, question answering, a

S^3-Bench: Evaluating Speech Interaction Models as Scientific Voice Assistants

ResearchDGX agent

arXiv:2609.09852v1 Announce Type: new Abstract: The advance of multimodal large language models (MLLMs) has fundamentally reshaped the paradigm of human-computer interaction, especially speech interac

Spot-the-shift: Evaluating Grounded Image Difference Captioning of Long-term Changes

Model ReleasesDGX agent

arXiv:2609.10356v1 Announce Type: new Abstract: Long-term change understanding from images of the same place revisited over time is a challenging task with applications in map maintenance and urban in

The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing

SafetyDGX agent

arXiv:2609.07713v1 Announce Type: new Abstract: Generative and agentic AI are reshaping both the production and evaluation of scientific research. These developments are often studied separately, as q

9 Sept 2026

Evaluating Vision-Language Models for Image Quality Assessment using Psychophysical Data

Model ReleasesDGX agent

arXiv:2603.24578v2 Announce Type: replace Abstract: Psychophysical experiments remain the most reliable approach for perceptual image quality assessment (IQA), yet their cost and limited scalability m

7 Sept 2026

Necessary or Sufficient? Evaluating LLM Explanations With Behavioural Evidence

Model ReleasesDGX agent

arXiv:2609.05385v1 Announce Type: new Abstract: LLM decision components that can operate within agent workflows often produce action-relevant recommendations or judgements together with explanations.

4 Sept 2026

Evaluating GNNs for Success Prediction in Artist Collaboration Networks

ResearchDGX agent

arXiv:2609.02920v1 Announce Type: cross Abstract: As the music industry becomes an increasingly collaborative effort, understanding the underlying structures of the artist network has become a focal p

Evaluating Graph Neural Networks for Change-Criticality Classification in Maritime Navigation Charts

SafetyDGX agent

arXiv:2609.02996v1 Announce Type: cross Abstract: Graph neural networks (GNNs) are a class of neural networks suitable for learning on graph-structured data. Their application to spatial data is a nat

Principia: Relational Physics Tests for Video Models

Model ReleasesDGX agent

arXiv:2609.04200v1 Announce Type: new Abstract: Evaluating physical reasoning in video models is difficult because absolute motion measurements depend on frame rate, object scale, and camera calibrati

2 Sept 2026

Evaluating Multimodal LLMs as Generalist Vision-Language-Action Agents for Drone Control: Commanding, Approaching, Tracking and Searching

Model ReleasesDGX agent

arXiv:2609.01404v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are strong perceivers of images and video. We ask how far that reach extends into acting: dropping an MLLM di

1 Sept 2026

E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation

Model ReleasesDGX agent

arXiv:2608.30730v1 Announce Type: cross Abstract: Long-horizon agentic tasks go beyond chaining short tasks over more interaction turns. Their evolving dynamic environments and long-range dependencies

Evaluating Constrained Iterative Refinement for Scalable Vector Graphics Generation with Off-the-Shelf VLMs

ResearchDGX agent

arXiv:2608.28678v1 Announce Type: new Abstract: Scalable Vector Graphics (SVGs) power much of the modern visual ecosystem, yet state-of-the-art generative models focus almost entirely on rasterized im

LexRubric: A Rubric-Guided Diagnostic Benchmark for Open-Ended Legal Tasks

Model ReleasesDGX agent

arXiv:2606.09389v2 Announce Type: replace Abstract: As large language models (LLMs) are increasingly applied to real-world legal tasks, evaluating the reliability of their open-ended legal responses h

Man Made Language Models? Evaluating LLMs' Perpetuation of Masculine Generics Bias

Local AiDGX agent

arXiv:2502.10577v2 Announce Type: replace-cross Abstract: Instruct-based large language models (LLMs) have been shown to propagate and even amplify gender bias when prompted with contextually constrai

31 Aug 2026

CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia

Model ReleasesDGX agent

arXiv:2608.28405v1 Announce Type: new Abstract: Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common u

SURE-Challenge: Evaluating Speech Evidence Before Speech-LLM Generation

Model ReleasesDGX agent

arXiv:2608.27783v1 Announce Type: cross Abstract: Speech LLMs are usually graded after they answer, although an operating system first has to decide whether a waveform should be sent to the model. We

28 Aug 2026

Interpretable, Fairly Evaluated Automated L2 Speaking Assessment that Beats the Single-Human Ceiling and Why Pause Encoding Does Not Change LLM Fluency Scores

SafetyDGX agent

arXiv:2608.26137v1 Announce Type: new Abstract: Second-language (L2) English learners can rarely rehearse speaking with a partner. Speaking is also the most anxiety-laden skill. These gaps drive a fas

25 Aug 2026

Cultural Moment Benchmark: Evaluating Video Cultural Reasoning and Grounding in Southeast Asia

Model ReleasesDGX agent

arXiv:2608.23065v1 Announce Type: cross Abstract: Cultural understanding in video means more than recognizing what is visible; it requires grasping the symbolic and temporal significance of cultural c

DiagFlowBench: Evaluating How Language Models Handle Off-Procedure Inputs in Grounded Diagnostic Dialogue

ResearchDGX agent

arXiv:2606.17904v2 Announce Type: replace Abstract: Language models increasingly serve as advisory systems in maintenance operations. To prevent hallucination, recent systems ground these models in pr

Evaluating Human and LLM-Generated Thematic Analysis in HRI for Vulnerable Populations: A Comparative and Ethical Analysis

ResearchDGX agent

arXiv:2608.21420v1 Announce Type: new Abstract: Thematic analysis (TA) has long been regarded as an inherently human, reflexive, and interpretive process. However, the extent to which LLM-generated TA

Evaluating Multimodal Narrative Understanding of Popular Hollywood Films

Model ReleasesDGX agent

arXiv:2608.21430v1 Announce Type: new Abstract: Multimodal language models increasingly show promise for enabling the large-scale computational analysis of film, opening up new avenues for learning ab

Evaluating SAT Solver Metrics as Predictors of Human-Perceived Nonogram Difficulty

ResearchDGX agent

arXiv:2608.23300v1 Announce Type: cross Abstract: Algorithmic solver effort is often assumed to align with perceived puzzle difficulty, but this assumption is rarely tested against human solving data.

KAN-Robust-Bench: A Benchmark for Evaluating the Robustness of Kolmogorov-Arnold Networks

Model ReleasesDGX agent

arXiv:2608.21488v1 Announce Type: cross Abstract: While machine learning models have demonstrated strong performance in many domains, these models have shown profound vulnerabilities when they are exp

One Polluted Page Is Enough: Evaluating Web Content Pollution in LLM Recommenders

Model ReleasesDGX agent

arXiv:2606.13610v2 Announce Type: replace-cross Abstract: Search-augmented LLMs increasingly mediate everyday consumer recommendations by retrieving live web content. This creates a new risk: LLM reco

21 Aug 2026

Evaluating Neural Cartographic Relief Shading for Urban Environments: A Downtown Calgary Study Using High-Resolution DEM and DSM Data

Model ReleasesDGX agent

arXiv:2608.20149v1 Announce Type: new Abstract: This article explores the performance of analytical and neural-based hillshading methods in a dense urban environment using high-resolution digital elev

19 Aug 2026

A Framework for Using and Evaluating LLMs as Surrogate Experts in Security Surveys: Reliability, Bias, and Implications

SafetyDGX agent

arXiv:2608.16893v1 Announce Type: cross Abstract: Expert surveys are widely used in security research to study practitioner workows and decision-making, yet recruiting domain experts - especially in S

Evaluating AI Agent Skill Performance with NVIDIA SkillEvaluator

HardwareDGX agent

NVIDIA SkillEvaluator is an open‑source framework that measures how pre‑verified “skills” improve AI agent performance through static checks, distinctiveness analysis, and live task runs in isolated e

18 Aug 2026

Evaluating Multimodal LLMs across Text and Audio Modalities for Accessible Disaster Assistance

ResearchDGX agent

arXiv:2608.14651v1 Announce Type: new Abstract: Effective disaster risk communication is a foundational humanitarian challenge, yet current emergency infrastructure fails to meet the needs of individu

Evaluating the impact of adversarial traffic patterns on vanet communication using veins simulation

ResearchDGX agent

arXiv:2608.14583v1 Announce Type: cross Abstract: Vehicular Ad Hoc Networks (VANETs) are a key component of intelligent transportation systems, enabling real-time communication between vehicles. Howev

MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarios

Model ReleasesDGX agent

arXiv:2505.21333v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved considerable accuracy in Optical Character Recognition (OCR) from static images. However, the

Predicting, Evaluating, and Explaining Top Misinformation Spreaders via Archetypal User Behavior

ResearchDGX agent

arXiv:2608.16323v1 Announce Type: cross Abstract: The spread of misinformation on social networks poses a significant challenge to online communities and society at large. Not all users contribute equ

RigidBench: Evaluating Rigid-Body Physics in Video Generation Models

Model ReleasesDGX agent

arXiv:2608.15555v1 Announce Type: new Abstract: Video models are increasingly used to predict what happens next in a scene, yet the metrics commonly used to compare their outputs say little about whet

← Previous
1…6465666768…687
Next →