AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,652
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,793
  • Industry6,824
  • Local Ai7,051
  • Model Releases36,896
  • Research29,820
  • Safety19,754
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,652
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,793
  • Industry6,824
  • Local Ai7,051
  • Model Releases36,896
  • Research29,820
  • Safety19,754
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,652Total entries
60Added by human
127,592Found by agent
13Categories

Knowledge catalogue

Search: “evaluation”

GridTimelineEvolution
41,190 results
28 Jul 2026

ParBench: A Benchmark for Reliable Evaluation of LLM Parallel Code Translation

Model ReleasesDGX agent

arXiv:2607.22588v1 Announce Type: new Abstract: Modern compute-intensive software must migrate across a changing ecosystem of accelerators, programming APIs, compiler stacks, and portability layers, i

The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluation

Model ReleasesDGX agent

arXiv:2607.22585v1 Announce Type: new Abstract: Public leaderboards for coding agents typically rank systems by model name and pass rate, while the surrounding harness (the scaffold that issues tools,

Trustworthy Medical Segmentation: Uncertainty-Aware U-Net Evaluation Under Clinical Image Degradation

Model Releases
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
DGX agent

arXiv:2607.22727v1 Announce Type: new Abstract: Medical image segmentation models often report high benchmark accuracy under ideal imaging conditions, yet their failures under clinical degradation can

27 Jul 2026

Agentic Evaluation of Copyright Law Compliance

Model ReleasesDGX agent

arXiv:2607.21799v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly perform commercial tasks that involve retrieving external content such as images and, where appropriate,

Teachy Mini: Development and Preliminary Evaluation of a Knowledge-Based Generative Social Robot for Higher Education

ApplicationsDGX agent

arXiv:2607.22345v1 Announce Type: new Abstract: Generative social robots (GSRs) powered by large language models offer new possibilities for personalized tutoring in higher education, but also introdu

24 Jul 2026

A Comparative Evaluation of Embeddings and LLMs in a Greek Book Publisher Setting - The CUP Dataset

Model ReleasesDGX agent

arXiv:2607.21274v1 Announce Type: cross Abstract: We present CUP, a Greek book retrieval benchmark consisting of 868 catalog records and 104 expert-annotated queries with graded relevance judgments. W

Evaluation and Prognostic Validation of Deep Regression Models for WSI-Based Gene-Expression Prediction

ResearchDGX agent

arXiv:2410.00945v2 Announce Type: replace-cross Abstract: Gene-expression profiling is widely used in research and central to many areas of precision oncology, but remains costly and not universally a

Verifier-First Evaluation of Agentic LLMs for Infrastructure-as-Code Generation

Model ReleasesDGX agent

arXiv:2607.20478v1 Announce Type: cross Abstract: Infrastructure-as-Code (IaC) generation from natural language requires satisfying provider schemas, dependency planning, and organizational policy con

23 Jul 2026

A Multi-Dimensional Evaluation of Explainability in Media Bias Detection

SafetyDGX agent

arXiv:2607.19954v1 Announce Type: new Abstract: Detecting media bias automatically is difficult because biased framing is often subtle, yet in domains such as news analysis, accurate predictions alone

Attention Without Grounding: Causal Evaluation of Visual Explanations in Medical VLMs

ResearchDGX agent

arXiv:2607.18577v1 Announce Type: new Abstract: Attention and saliency heatmaps are widely used to explain medical Vision-Language Model (VLM) outputs on chest X-rays, yet whether they truly highlight

Crashing Waves vs. Rising Tides: Findings on AI Automation from Thousands of Worker Evaluations of Labor Market Tasks

ResearchDGX agent

arXiv:2604.01363v2 Announce Type: replace Abstract: We propose that AI automation is a continuum between: (i) crashing waves where AI capabilities surge abruptly over small sets of tasks, and (ii) ris

FORCE-Bench: A Benchmark, Dataset, and Evaluation Harness for Agentic AI in Enterprise Finance

Model ReleasesDGX agent

arXiv:2607.19409v1 Announce Type: new Abstract: Recent advances in large language models have accelerated deployment of agentic systems in operational finance. Existing benchmarks emphasize measuring

OpenAI–Hugging Face Model Evaluation Security Incident (Official Account)

SafetyHuman

OpenAI’s official incident page: eval agents exploited a zero-day to reach the internet and compromise Hugging Face during testing.

16 Jul 2026

Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System

AgentsDGX agent

arXiv:2607.13370v1 Announce Type: cross Abstract: This paper is an extension of a paper presented at the ICAART 2026 conference, which introduced LEA (Learning Engagement Assistant), an adaptive AI tu

Prospective clinical indication, post-hoc report leakage, and fusion design in multi-image chest radiograph classification: a patient-clustered evaluation

ResearchDGX agent

arXiv:2607.13800v1 Announce Type: cross Abstract: Chest radiograph datasets often combine multiple images with Clinical Indication, Findings, and Impression, although these inputs are produced at diff

15 Jul 2026

Amplitude-Only FFN Intervention for Tool-Structured LLM Inference Method: Gated Evaluation Protocol, and Cross-Model Empirical Results

AgentsDGX agent

arXiv:2607.11183v2 Announce Type: replace Abstract: Large language models increasingly operate as tool-using agents, where small format, argument, or function-call errors can invalidate otherwise plau

Calibrated Selective Prediction Using Deep Ensembles for ROI-Based Thyroid Nodule Ultrasound Classification Under Dataset Shift: A Retrospective Evaluation

SafetyDGX agent

arXiv:2607.12075v1 Announce Type: cross Abstract: Background: Deep learning models can classify thyroid nodules on ultrasound, but reliable clinical decision support also requires calibrated probabili

Form, Not Content? A Preregistered, Placebo-Controlled Evaluation of Learned Error-Conditioned Self-Repair Through Prompts and Weights in Frozen Small Code Models

Local AiDGX agent

arXiv:2607.12962v1 Announce Type: cross Abstract: Frozen small code LLMs are deployed locally, yet the information guiding a retry after a failed attempt is still measured without placebo controls in

10 Jul 2026

Metrics or Mirage? An Audit of Evaluation Inconsistencies in Colonoscopy Polyp Segmentation Benchmarks

ResearchDGX agent

arXiv:2607.08203v1 Announce Type: new Abstract: Progress in colonoscopy polyp segmentation is routinely reported through leaderboard comparisons on a small set of public benchmarks. We argue that this

Principled Analysis of Deep Reinforcement Learning Evaluation and Design Paradigms

ResearchDGX agent

arXiv:2607.07769v1 Announce Type: cross Abstract: Starting from the utilization of deep neural networks to approximate the state-action value function that led to winning one of the most challenging g

Systematic Evaluation of Learning Rate Scheduling Strategies Across Heterogeneous Architectures

ResearchDGX agent

arXiv:2607.08511v1 Announce Type: cross Abstract: Choosing a learning rate scheduling strategy is critical to neural network training, but manual selection is costly and rarely exhaustive. While class

VEGAS: Human-Aligned Video Caption Evaluation via Gaze

ResearchDGX agent

arXiv:2607.08489v1 Announce Type: cross Abstract: Vision-language models excel at video captioning, yet typically generate descriptions that fail to capture individual viewers' attention. We propose V

9 Jul 2026

AgentLens: Production-Assessed Trajectory Reviews for Coding Agent Evaluation

Model ReleasesDGX agent

arXiv:2607.06624v1 Announce Type: new Abstract: We present AgentLens, a production-assessed benchmark for interactive code agents. Most code-agent benchmarks reduce a run to a single bit -- did the ta

Discovering Geometric Biases in 3D Face Reconstruction: A Curvature-Aware Spectral Framework for Fairness Evaluation

SafetyDGX agent

arXiv:2607.07486v1 Announce Type: new Abstract: 3D Morphable Models (3DMMs) remain the standard parametric shape priors for many state-of-the-art 3D face reconstruction algorithms. However, as these m

Overview of the NLPCC 2026 Shared Task 1: Difficulty-Aware Multilingual and Multimodal Medical Instructional Video Understanding Evaluation

Model ReleasesDGX agent

arXiv:2607.06618v1 Announce Type: cross Abstract: Following the CMIVQA, MMI-VQA, and M4IVQA challenges in NLPCC 2023--2025, we introduce the Difficulty-Aware Medical Instructional Video Question Answe

8 Jul 2026

A VLM-Enhanced Framework for Comprehensive Traffic Sign Condition Assessment Integrating Daytime Visual Performance and Nighttime Retroreflectivity Evaluation

Model ReleasesDGX agent

arXiv:2607.06478v1 Announce Type: new Abstract: Traffic signs are crucial components of road safety, serving as visual tools under all lighting conditions. The Manual on Uniform Traffic Control Device

On the feasibility of dependency parsing of non-human sequences without a gold standard. Is evaluation possible in other species?

ResearchDGX agent

arXiv:2607.06542v1 Announce Type: new Abstract: Dependency parsing consists of finding a tree representation for a sequence. Unsupervised dependency parsing aims to develop parsing methods without a g

PIPBench: A Profile-Inclusive Framework for Personalized Image Generation Evaluation

Model ReleasesDGX agent

arXiv:2607.06440v1 Announce Type: new Abstract: Recent text-to-image models such as DALLE-3 excel at following diverse prompts yet remain blind to individual aesthetic preferences. We study personaliz

7 Jul 2026

Graph Classification via Network Usable Information: From Representation Evaluation to Structure Selection

Model ReleasesDGX agent

arXiv:2607.03587v1 Announce Type: new Abstract: We propose NetinfoGC, a framework for graph classification that extends the Network Usable Information (NUI) paradigm to graph-level learning. Unlike co

RLIE: Rule Generation with Logistic Regression, Iterative Refinement, and Evaluation for Large Language Models

TutorialsDGX agent

arXiv:2510.19698v3 Announce Type: replace Abstract: Large Language Models (LLMs) can propose rules in natural language, sidestepping the need for a predefined predicate space in traditional rule learn

SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models

Model ReleasesDGX agent

arXiv:2607.05365v1 Announce Type: cross Abstract: Streaming speech-to-speech language models aim to answer spoken queries directly with synthetic speech. However, standard speech and text benchmarks d

Towards Standardized Light Field Quality Assessment: Hybrid Subjective Benchmarking and Objective Metric Evaluation

Model ReleasesDGX agent

arXiv:2607.03494v1 Announce Type: new Abstract: Benchmarking immersive media coding solutions, especially in the standardization context, requires reliable and reproducible subjective quality assessme

TRIAGE: Trustworthy Retrieval Instrumentation And Graph Evaluation

ResearchDGX agent

arXiv:2607.03447v1 Announce Type: cross Abstract: Knowledge graphs (KGs) that underpin Graph-based Retrieval-Augmented Generation (Graph-RAG) are increasingly built automatically by LLM-driven extract

3 Jul 2026

Prompt Framing Distorts Count-Based Evaluation of LLM Error Detection: Evidence from Numeric Anchoring

Model ReleasesDGX agent

arXiv:2607.01240v1 Announce Type: cross Abstract: Count-based F1 is widely used as a proxy for LLM error-detection quality, but this paper shows that it can rise dramatically without a corresponding i

Psychological Steering in LLMs: An Evaluation of Effectiveness and Trustworthiness

Model ReleasesDGX agent

arXiv:2510.04484v2 Announce Type: replace-cross Abstract: The ability to control LLMs' emulated emotional states and personality traits is an essential step in enabling rich, human-centered interactio

2 Jul 2026

Sheet Music Benchmark: Standardized Optical Music Recognition Evaluation

Model ReleasesDGX agent

arXiv:2506.10488v3 Announce Type: replace Abstract: In this work, we introduce the Sheet Music Benchmark (SMB), a dataset of six hundred and eighty-five pages specifically designed to benchmark Optica

Vibe Coding Ate My Homework: An evaluation of AI approaches to greenfield software engineering and programming

ResearchDGX agent

arXiv:2606.18293v2 Announce Type: replace-cross Abstract: Thanks to rapid developments in generative AI, we are in the midst of a paradigm shift that may change how we interact with computers forever.

1 Jul 2026

A Realistic Protocol for Evaluation of Weakly Supervised Object Localization

Model ReleasesDGX agent

arXiv:2404.10034v3 Announce Type: replace Abstract: Weakly Supervised Object Localization (WSOL) allows training deep learning models for classification and localization (LOC) using only global class-

Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian

Model ReleasesDGX agent

arXiv:2606.31718v1 Announce Type: cross Abstract: Relation extraction (RE) for low-resource languages is typically constrained by the lack of annotated corpora. We investigate the feasibility of cross

Patient-Level Elbow Abnormality Detection: Leakage-Aware Evaluation of Learned Preprocessing, Calibration, and Triage-Oriented Operating Points

ResearchDGX agent

arXiv:2606.31348v1 Announce Type: new Abstract: In this study, we examine learned preprocessing pipelines in the context of triage-oriented orthopedic abnormality detection task using elbow radiograph

Probing Stylistic Appropriation using Large Language Models: An Evaluation Framework for Copyright Infringement under EU Law

Model ReleasesDGX agent

arXiv:2606.31250v1 Announce Type: cross Abstract: Large language models (LLM) trained on web-scale corpora generate output that may infringe copyright, yet existing technical safeguards focus narrowly

Wind and State Estimation on SE(3): Comparative Evaluation of EKF and UKF with Continuous and Discrete Quadrotor Models

ResearchDGX agent

arXiv:2606.30804v1 Announce Type: new Abstract: Use of quadrotor UAVs for wind velocity estimation is gaining popularity in recent studies, leveraging their maneuverability, compact size and low cost.

30 Jun 2026

Clinical Reasoning Graphs: Structured Evaluation of LLM Diagnostic Reasoning Reveals Competence Without Consistency

ResearchDGX agent

arXiv:2606.29876v1 Announce Type: cross Abstract: Modern large language models (LLMs) reach 60-70% diagnostic accuracy on complex clinical case benchmarks, but accuracy alone cannot distinguish stable

Empirical Evaluation of Multi-Modal Touch Detection in Over-the-Shoulder Video Surveillance

ResearchDGX agent

arXiv:2606.29504v1 Announce Type: new Abstract: Video Intelligence Surveillance (VIDINT) on over-the-shoulder footage is a proposed vector for monitoring human-computer interaction patterns without di

Overcoming Dependent Censoring in the Evaluation of Survival Models

ResearchDGX agent

arXiv:2502.19460v4 Announce Type: replace-cross Abstract: Dependent censoring occurs when the event time and censoring time are not conditionally independent given the observed covariates. This compli

SABER-Math: Automated Benchmark for Information Retrieval Evaluation in Mathematics

Model ReleasesDGX agent

arXiv:2606.29894v1 Announce Type: cross Abstract: As agentic AI systems tackle more complex mathematical tasks, they increasingly rely on information retrieval (IR) to search problem databases, theore

26 Jun 2026

ProfileFoundry: A Synthetic Person-Object Substrate for Privacy, Memory, and Tool-Use Evaluation in LLM Agent

Local AiDGX agent

arXiv:2606.26403v1 Announce Type: new Abstract: Foundation-model research increasingly needs data about people: user state, personal histories, relationships, contact-like fields, documents, and longi

25 Jun 2026

Evaluation Protocols and Validation for Cameras in Indoor Healthcare Monitoring

SafetyDGX agent

arXiv:2606.25284v1 Announce Type: new Abstract: Camera-based monitoring systems are increasingly adopted in healthcare settings for the continuous assessment of patient movement and activities. Howeve

Gastroendoscopy View Synthesis: A New Real Dataset and Evaluation

ResearchDGX agent

arXiv:2606.25427v1 Announce Type: new Abstract: Novel view synthesis (NVS) is an active research topic in computer vision, owing to the success of neural radiance field (NeRF) and 3D Gaussian splattin

24 Jun 2026

A Fair Evaluation of Graph Foundation Models for Node Property Prediction

ApplicationsDGX agent

arXiv:2606.24509v1 Announce Type: cross Abstract: Due to the wide use of graph-structured data in different fields of industry and science, the development of Graph Foundation Models (GFMs) has recent

Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs

ResearchDGX agent

arXiv:2606.23915v1 Announce Type: new Abstract: Practice often treats automatic metrics for attribution in LLM retrieval-augmented generation as interchangeable. We audit eight automatic scorers -- le

One Ruler: A Same-Hands Re-Evaluation of Bivariate Causal Direction on Tuebingen, with a Parameter-Free Compression Baseline

Model ReleasesDGX agent

arXiv:2606.23767v1 Announce Type: new Abstract: Headline accuracies on the Tuebingen cause-effect pairs are routinely compared across papers even though each is measured under its authors' own protoco

Trimming the Long-Tail of Visual World Modeling Evaluation

Model ReleasesDGX agent

arXiv:2606.24256v1 Announce Type: new Abstract: Physical interactions follow a long-tailed distribution: a set of common and regular interactions dominates human experience and visual data, while a br

23 Jun 2026

Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation

Model ReleasesDGX agent

arXiv:2606.15127v2 Announce Type: replace Abstract: Reasoning models are increasingly used in settings where the final answer is not the only object of review: educational tools may show students inte

ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures

SafetyDGX agent

arXiv:2606.19380v2 Announce Type: replace-cross Abstract: Software engineering and deployment are increasingly delegated to AI coding agents. The scale of their adoption is surfacing rare, but highly

Each Judge Its Own Yardstick: Discovering Per-VLM Taxonomies for Physical Video Evaluation

Model ReleasesDGX agent

arXiv:2606.22918v1 Announce Type: new Abstract: Maintaining physical consistency in video generators and world models increasingly relies on vision-language models (VLMs) as automated judges that prov

Evaluation of Medical Vision Language Models HuluMed and MedGemma, and general purpose chatbots Gemma 3, ChatGPT Plus, and Claude Pro on real previously unseen wound images

Model ReleasesDGX agent

arXiv:2606.20723v1 Announce Type: new Abstract: Chronic wound assessment remains a clinically challenging task that requires accurate interpretation of wound morphology, tissue composition, vascular c

REKEY: Metadata-Grounded Visual-Key Regeneration for Contamination-Resilient VQA Evaluation

Model ReleasesDGX agent

arXiv:2606.20736v1 Announce Type: new Abstract: Static visual question answering (VQA) benchmarks age quickly: Once the items leak into training corpora, scores can reflect memorization rather than ge

11 Jun 2026

Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application

AgentsDGX agent

arXiv:2606.12191v1 Announce Type: cross Abstract: Environments serve as interactive systems for large language model (LLM) based agents across diverse scenarios and play a crucial role in driving the

From Nominal Intensity to Equivalent Rainfall: A Path-Based Credibility Evaluation Framework for Simulated Rainfall in Autonomous-Driving Perception Tests

AgentsDGX agent

arXiv:2606.11989v1 Announce Type: new Abstract: Credible simulated-rainfall conditions are essential for identifying perception-system boundaries and supporting SOTIF-oriented risk assessment in autom

← Previous
1…3031323334…687
Next →