AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “evaluation”

GridTimelineEvolution
41,190 results
7 Aug 2026

Evaluating Machine Learning Models for Post-Wildfire Debris-Flow Prediction

ResearchDGX agent

arXiv:2608.05265v1 Announce Type: cross Abstract: Prediction of post-wildfire debris flows is critical for mitigating hazards to communities, infrastructure, and resources during intense rainfall in r

FormBharo: Designing and Evaluating a Voice Agent for Conversational Form Filling in Rural India

Model ReleasesDGX agent

arXiv:2608.06027v1 Announce Type: cross Abstract: In India, almost every social benefit starts with a form, yet the people who need these benefits most are often unable to read or write. Reaching them

Innocent Panels, Hateful Stories: Evaluating and Detecting Hateful Intent in Multi-Turn Visual Story Generation

Model ReleasesDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2608.05210v1 Announce Type: cross Abstract: Picture books and comics have long been used to disseminate hateful narratives because they are easily understood even by children, as exemplified by

6 Aug 2026

Evaluating the Diagnostic Robustness of Vision-Language Models Under Visual and Textual Perturbations

SafetyDGX agent

arXiv:2608.04885v1 Announce Type: cross Abstract: Standard accuracy metrics for VLMs often mask significant reliability failures in sensitive domains. In this work, we utilize a histopathology-validat

Global Attention-Fused Image Cropping with Attention-Guided and Global-Aligned Crop Evaluator

ResearchDGX agent

arXiv:2608.04821v1 Announce Type: new Abstract: Image cropping aims to improve image aesthetics by preserving important content within an appropriately composed region. However, most existing methods

IslamicTurathBench: A Multi-Task, Multi-Discipline Benchmark for Evaluating Large Language Models on the Islamic Scholarly Tradition (turath)

Model ReleasesDGX agent

arXiv:2608.04703v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for question answering, education, and research, including in religious and cultural domains where an

The Evaluator Is Part of the Experiment: Measuring Open-Ended LLM Conformity

Model ReleasesDGX agent

arXiv:2608.04463v1 Announce Type: new Abstract: Prior work on LLM conformity largely measures discrete answer flips under verifiable labels. Open-ended revisions require a different measurement strate

5 Aug 2026

ChartAnno: Evaluating MLLMs for Chart Annotation Generation

Model ReleasesDGX agent

arXiv:2608.03464v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have made significant progress in chart understanding, generation, and editing, but their ability to annotate e

4 Aug 2026

Asleep at the Wheel: JEPA's Limitations in Evaluating Novel Driving Data

Model ReleasesDGX agent

arXiv:2608.01336v1 Announce Type: new Abstract: Modern autonomous-driving fleets record far more video than human reviewers can inspect. This motivates the need for an automatic clip triage mechanism,

MetaRoute-Bench: Evaluating Meta-Decision Policies for Agentic Workflow Routing

Model ReleasesDGX agent

arXiv:2608.00107v1 Announce Type: new Abstract: Agentic systems must repeatedly decide whether to answer directly, decompose a task, invoke a tool, execute code, delegate to a specialist, verify an in

Style Wins, Substance Loses: A Diagnosis of LLM-as-Judge in Idea Generation

Model ReleasesDGX agent

arXiv:2608.01666v1 Announce Type: new Abstract: However, whether these judges truly evaluate the scientific substance of ideas or are influenced by superficial stylistic presentation remains an open q

3 Aug 2026

FairFund-Bench: Evaluating Distributive Bias in LLM Resource Allocation

Model ReleasesDGX agent

arXiv:2607.28934v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly involved in the distribution of scarce resources, raising concerns about biased allocations based on cha

31 Jul 2026

JigShape: Evaluating Visual-Geometric Reasoning in VLMs through Jigsaw Puzzles

Model ReleasesDGX agent

arXiv:2607.27670v1 Announce Type: new Abstract: Jigsaw puzzle solving requires jointly reasoning about visual content and geometric constraints, yet existing benchmarks use rectangular cuts that creat

29 Jul 2026

Evaluating VLMs for Autonomous Agent-Driven Geometry Clipping Detection in Video Game QA

Model ReleasesDGX agent

arXiv:2607.25921v1 Announce Type: cross Abstract: In this work, we study the use of Vision-Language Models (VLMs) for anomaly detection in an agent-driven game Quality Assurance (QA) pipeline focusing

MEDit-Bench: A Dataset for Evaluating Message-Driven Narrative Video Editing

Model ReleasesDGX agent

arXiv:2607.25300v1 Announce Type: new Abstract: Video editing is fundamentally message-driven: even from the same source footage, the selected shots change depending on the narrative the editor wishes

MyoCardBench: A Real-World Data Benchmark for Evaluating Large Language Models in Clinically Authentic Cardiovascular Care Scenarios

Model ReleasesDGX agent

arXiv:2607.25186v1 Announce Type: new Abstract: Background: Most medical large language model (LLM) benchmarks focus on examination knowledge or isolated tasks and may not reflect the longitudinal, mu

Polistemics: Evaluating LLMs as Information Mediators in Politics & Elections

Model ReleasesDGX agent

arXiv:2607.25953v1 Announce Type: new Abstract: As LLMs increasingly mediate the political information citizens rely on, there is still no standardized way to assess whether they do so responsibly. We

28 Jul 2026

Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests

ResearchDGX agent

arXiv:2607.22883v1 Announce Type: cross Abstract: While Large Language Models (LLMs) show great promise for automating unit test generation, recent studies suggest that the quality of generated tests

Evaluating Fuzz Testing for Reinforcement Learning Agents

Model ReleasesDGX agent

arXiv:2607.24577v1 Announce Type: new Abstract: Reinforcement Learning (RL) agents are increasingly deployed in safety-critical domains such as robotics, autonomous driving, and drone control, where u

24 Jul 2026

Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks

Model ReleasesDGX agent

arXiv:2607.21482v1 Announce Type: new Abstract: Large language models (LLMs) and agents are now widely used tools in code development, with data typically sent to third-party cloud-based models. Their

ConfidenceBench: Evaluating Confidence Calibration in Large Language Models

Model ReleasesDGX agent

arXiv:2607.20526v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in settings where fluent but incorrect answers can be costly. In these settings, accuracy alone i

Evaluating Risks in Weak-to-Strong Alignment: A Bias-Variance Perspective

SafetyDGX agent

arXiv:2604.25077v2 Announce Type: replace Abstract: Weak-to-strong alignment offers a promising route to scalable supervision, but it can fail when a strong model becomes confidently wrong on examples

SciExplore: Evaluating Autonomous Agents from Scientific Navigation to Information Integration

Model ReleasesDGX agent

arXiv:2607.20926v1 Announce Type: new Abstract: Scientific research involves complex information-seeking and reasoning workflows across heterogeneous sources. However, existing benchmarks primarily em

SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents

Model ReleasesDGX agent

arXiv:2607.15557v4 Announce Type: replace Abstract: Agent skills, SKILL files that package reusable procedural knowledge for an LLM agent, are a popular mechanism for extending agent capabilities. Pub

23 Jul 2026

It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief

Model ReleasesDGX agent

arXiv:2607.18232v2 Announce Type: replace Abstract: Users frequently express their beliefs to large language models (LLMs). In some situations, the LLM should accept these contextual beliefs as true.

MeetingToM: Evaluating Multimodal LLMs on Theory-of-Mind Reasoning in Multi-Party Meetings

Model ReleasesDGX agent

arXiv:2607.19235v1 Announce Type: cross Abstract: Theory of Mind (ToM), the ability to infer other's beliefs, intentions, and states of knowledge, is central to social interaction, yet remains challen

15 Jul 2026

Evaluating Reliability in Machine Learning Models for Early Chronic Kidney Disease Prediction: A Systematic Review of Data Leakage and Predictor Stability

ApplicationsDGX agent

arXiv:2607.11963v1 Announce Type: cross Abstract: The early detection of Chronic Kidney Disease using machine learning has attracted significant interest in healthcare-related computer science. Despit

LLMs Can See the Smoke but not the Fire: Evaluating Abductive Reasoning with Elenchos

ResearchDGX agent

arXiv:2607.12733v1 Announce Type: new Abstract: Large language models (LLMs) excel at pattern recognition and text generation, but their capacity for abductive inference - inferring latent hypotheses

10 Jul 2026

3 production patterns for AI agents and how to evaluate each one

Local AiDGX agent

A local coding agent, an in-app customer assistant, and an AI SRE triaging production logs may all use the same model class—but not the same harness, eval plan, or rollout risk. Mastra CEO Sam Bhagwat

Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Models

Model ReleasesDGX agent

arXiv:2607.08317v1 Announce Type: new Abstract: Modern AI models achieve strong performance on many established benchmarks, yet they still fail on tasks that humans find almost trivial, such as manipu

MentalHospital: A Virtual Environment for Evaluating Psychiatric Clinical Encounters

SafetyDGX agent

arXiv:2607.08257v1 Announce Type: new Abstract: Large language models (LLMs) have shown strong performance on isolated psychiatric tasks, including dialogue, diagnosis, and treatment planning, yet exi

9 Jul 2026

Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematics

Model ReleasesDGX agent

arXiv:2607.06820v1 Announce Type: new Abstract: Recent advances in AI for Mathematics have focused largely on autoformalization and theorem proving, leaving the role of Computer Algebra Systems (CAS)

8 Jul 2026

Beyond Accuracy: How Humans Evaluate Legally Correct but Socially Controversial Legal Advice from Machines

ApplicationsDGX agent

arXiv:2607.05680v1 Announce Type: cross Abstract: AI systems are increasingly used to provide legal advice, raising questions about whether laypeople accept guidance from algorithms--especially when t

Faithful or Findable? Evaluating LLM-Generated Metadata for RDF Dataset Search

AgentsDGX agent

arXiv:2607.05970v1 Announce Type: cross Abstract: Dataset search depends heavily on metadata, making LLM-generated metadata a consequential form of synthetic content in retrieval systems. We study six

7 Jul 2026

ClassicLogic: A Knowledge-Driven Benchmark of Classic Puzzle Games for Evaluating Compositional Generalization

Model ReleasesDGX agent

arXiv:2607.05185v1 Announce Type: new Abstract: Compositional generalization, the ability to understand and produce novel combinations of known components, remains a fundamental challenge for modern a

Criterion-Conditional In-Context Learning: Evaluating Criterion-Shift Adaptation in Vision-Language Models

Model ReleasesDGX agent

arXiv:2607.02575v1 Announce Type: cross Abstract: Vision-language models can perform new tasks without parameter updates through in-context learning (ICL), whose core mechanism is utilizing the suppor

Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards

Model ReleasesDGX agent

arXiv:2607.04727v1 Announce Type: cross Abstract: Automatic data visualization generation has advanced rapidly with multi-modal large language models, yet existing efforts largely focus on static char

Evaluating and Understanding Model Editing for Medical Vision Language Models

Model ReleasesDGX agent

arXiv:2607.05310v1 Announce Type: new Abstract: Model editing promises a fast, targeted way to correct post-deployment mistakes in medical vision-language models (VLMs) without costly retraining. Howe

Evaluating Intellectual Property Guardrails of Generative Image Models: A Technical Report

Model ReleasesDGX agent

arXiv:2607.02582v1 Announce Type: new Abstract: Generative image models are capable of producing images that bear a strong resemblance to, or replicate, recognizable intellectual property (IP). In thi

Evaluating Large Language Models for Antisemitic Incident Classification

Model ReleasesDGX agent

arXiv:2607.04890v1 Announce Type: new Abstract: Addressing hate and violence in society requires timely detection of hateful events from public reporting, but automated identification of hateful event

Evaluating LLM Uncertainty in Long-Form Generation Using Deterministic Ground Truth

Model ReleasesDGX agent

arXiv:2607.03870v1 Announce Type: new Abstract: As LLMs generate increasingly long outputs, effective uncertainty estimation must identify errors at fine-grained levels rather than discard entire resp

SovereignNegotiation-Bench: Evaluating User-Owned Personal Agents In Delegated Bargaining Under Privacy, Consent, Evidence, And Institutional Pressure

Model ReleasesDGX agent

arXiv:2607.02814v1 Announce Type: cross Abstract: Personal agents will increasingly negotiate on behalf of users: splitting costs with other personal agents, appealing platform decisions, escalating s

VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs

Model ReleasesDGX agent

arXiv:2511.20272v2 Announce Type: replace Abstract: While Multimodal Large Language Models (MLLMs) have become adept at recognizing objects, they often lack the intuitive, human-like understanding of

3 Jul 2026

EduArt: An educational-level benchmark for evaluating art history knowledge in large language models

Model ReleasesDGX agent

arXiv:2607.02007v1 Announce Type: new Abstract: Large language models now score near ceiling on general benchmarks, but these aggregate measures reveal little about how models behave within single dis

Evaluating Chunking Strategies for Retrieval-Augmented Generation on Academic Texts

ResearchDGX agent

arXiv:2607.01852v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems use the question-answering capabilities of Large Language Models (LLMs) to access information outside the

Probabilistic Low-Voltage Peak Load Forecasting with Time Series Foundation Models Evaluated on Application-Oriented Metrics

ApplicationsDGX agent

arXiv:2607.01966v1 Announce Type: new Abstract: Low-voltage load forecasting is an important component in current and future energy systems with a high degree of electrification and decentralized gene

2 Jul 2026

EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards

Model ReleasesDGX agent

arXiv:2607.00218v1 Announce Type: cross Abstract: Vision-language models (VLMs) are now proposed as runtime safety guards for embodied agents in homes and factories. A deployable guard must catch genu

From 'Strings' to 'Things' for Personal Knowledge Graphs: Evaluating LLM Triple Extraction for Recommendation Systems

Model ReleasesDGX agent

arXiv:2607.00003v1 Announce Type: cross Abstract: Personal Knowledge Graphs (PKGs) offer a privacy-preserving framework for modeling user preferences, yet constructing them from unstructured, decentra

1 Jul 2026

CDR-Bench: Evaluating Faithful Execution of Compositional, Order-Sensitive Data Refinement Recipes

Model ReleasesDGX agent

arXiv:2606.31435v1 Announce Type: new Abstract: Data refinement involves executing multi-step recipes over evolving text states, where both composition and execution order of processing operators dete

Indi-RomCoM: Code-Mixed Benchmark for Evaluating LLMs on Romanized Indic-English Instructions

Model ReleasesDGX agent

arXiv:2606.30790v1 Announce Type: cross Abstract: Romanized Code Mixing (RCM), where bilingual speakers fluidly blend local languages with English in Roman script, has emerged as the dominant form of

Loc2Repair: A Framework for Evaluating the Impact of File-Level Issue Localization in Repo-Level LLM Repair

Local AiDGX agent

arXiv:2606.30963v1 Announce Type: cross Abstract: Repository-grounded automated repair is often reported as a single end-to-end capability, which hides distinct failure modes such as poor file targeti

Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens

Model ReleasesDGX agent

arXiv:2606.30755v1 Announce Type: cross Abstract: Claw-like AI agents (e.g., OpenClaw) are always-on processes with persistent access to credentials, files, tools, and external services. They take on

30 Jun 2026

Few-class Fidelity: Evaluating Explanations of Real-conditions CNN classifiers with Optimized Perturbations

Local AiDGX agent

arXiv:2606.28391v1 Announce Type: cross Abstract: The wide use of Convolutional Neural Networks (CNN) in numerous domains and real-world classification applications is justified by their high precisio

LLMs and their Limited Theory of Mind: Evaluating Mental State Annotations in Situated Dialogue

ResearchDGX agent

arXiv:2509.02292v2 Announce Type: replace Abstract: What if large language models could not only infer human mindsets but also expose every blind spot in team dialogue such as discrepancies in the tea

mamabench and mamaretrieval: Benchmarks for Evaluating Medical Retrieval-Augmented Generation in Maternal, Neonatal, and Reproductive Health

Model ReleasesDGX agent

arXiv:2606.29467v1 Announce Type: new Abstract: Medical question-answering benchmarks rarely cover the maternal, neonatal, child, and reproductive-health questions a nurse-midwife asks, and, to our kn

SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows

Model ReleasesDGX agent

arXiv:2606.29955v1 Announce Type: cross Abstract: Spreadsheets are widely used for business analysis, financial modeling, reporting, and decision-making. However, most existing spreadsheet benchmarks

Towards Evaluating Data Priors for Tabular Foundation Models

ResearchDGX agent

arXiv:2606.29241v1 Announce Type: new Abstract: Data-generating priors are a central component of tabular foundation models because they define the task distribution used during pretraining. However,

25 Jun 2026

Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?

AgentsDGX agent

arXiv:2602.11988v2 Announce Type: replace-cross Abstract: A widespread practice in software development is to tailor coding agents to repositories using context files, such as AGENTS.md. Although this

Evaluating LLMs on Real-World Software Performance Optimization

Model ReleasesDGX agent

arXiv:2606.25530v1 Announce Type: cross Abstract: Software performance optimization is a notoriously complex and manual task. Despite the growing use of Large Language Models (LLMs) for code refinemen

From Sounds to Scenes: A Benchmark for Evaluating Context-Aware Auditory Scene Understanding in Large Audio Language Models

Model ReleasesDGX agent

arXiv:2606.25391v1 Announce Type: cross Abstract: Recent Large Audio Language Models (LALMs) have achieved remarkable progress in audio perceptual tasks across individual acoustic layers, including sp

← Previous
1…6061626364…687
Next →