AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
1 Jun 2026

Translation Analytics for Freelancers II: Benchmarking Local LLMs for Confidential Translation Workflows

Model ReleasesDGX agent

arXiv:2605.31452v1 Announce Type: new Abstract: Building on our previous work, this paper develops practical, low-barrier methods for freelance translators and smaller language service providers to ev

29 May 2026

Benchmarking Large Vision-Language Models on CFMME: A Comprehensive Chinese Financial Multimodal Evaluation Dataset

Model ReleasesDGX agent

arXiv:2605.29462v1 Announce Type: cross Abstract: The emergence of Large Vision-Language Models (LVLMs) has substantially expanded model capabilities beyond text-only understanding, enabling unified i

Benchmarking LLM-Assisted Blue Teaming via Standardized Threat Hunting

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Model ReleasesDGX agent

arXiv:2509.23571v3 Announce Type: replace-cross Abstract: As cyber threats continue to grow in scale and sophistication, blue team defenders increasingly require advanced tools to proactively detect a

Benchmarking Positional Encoding Strategies for Transformer-Based EEG Foundation Models

Model ReleasesDGX agent

arXiv:2605.29754v1 Announce Type: new Abstract: Electroencephalography (EEG) is a widely used non-invasive technique for measuring brain activity in brain-computer interface (BCI) applications. Superv

Benchmarking Single-Factor Physical Video-to-Audio Generation

Model ReleasesDGX agent

arXiv:2605.30339v1 Announce Type: new Abstract: Generative video-to-audio (V2A) models produce highly plausible soundtracks, but it remains unclear whether they capture the underlying physical process

BenchTrace: A Benchmark for Testing Reflection Ability and Controlled Evolution in LLM Agents

Model ReleasesDGX agent

arXiv:2605.29225v1 Announce Type: new Abstract: Self-evolving agents improve over time by reflecting on past failures, but existing evaluation is limited in two ways: it measures only task scores, lea

Building and Road Recognition in Dense Urban Informal Settlements: A Dataset and Benchmark

Model ReleasesDGX agent

arXiv:2605.29856v1 Announce Type: new Abstract: As a widespread form of informal settlements, urban villages present significant challenges for sustainable urban development and governance. Precise ma

CLUBench: A Clustering Benchmark

Model ReleasesDGX agent

arXiv:2605.29933v1 Announce Type: new Abstract: Clustering is a fundamental problem in data science with a long-standing research history, yielding numerous insightful algorithms. Despite this progres

Converted, Not Equivalent: Benchmarking Codebase Conversion via Observational Equivalence

Model ReleasesDGX agent

arXiv:2605.29054v1 Announce Type: cross Abstract: Coding agents increasingly act as codebase-scale collaborators that can assist with codebase conversion, but this progress has exposed a critical weak

Do Physics Foundation Models Learn Generalizable Physics? A Bias-Aware Benchmark Across Physical Regimes and Distribution Shifts

Model ReleasesDGX agent

arXiv:2605.29283v1 Announce Type: cross Abstract: Recent physics foundation models claim general spatiotemporal forecasting ability, yet their evaluations often collapse performance into a single aver

GEO-Bench: Benchmarking Ranking Manipulation in Generative Engine Optimization

Model ReleasesDGX agent

arXiv:2605.29107v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly rank products, documents, and recommendations for user queries, which makes manipulating these rankings a gr

LoCoT2V-Bench: Benchmarking Long-Form and Complex Text-to-Video Generation

Model ReleasesDGX agent

arXiv:2510.26412v3 Announce Type: replace-cross Abstract: Recent advances in text-to-video generation have achieved impressive performance on short clips, yet evaluating long-form generation under com

LogDx-CI: Benchmarking Log Reduction Tools for LLM Root-Cause Diagnosis

Model ReleasesDGX agent

arXiv:2605.28876v1 Announce Type: cross Abstract: CI failure logs are large (median 5k lines, max 200k in this corpus) and noisy. Coding agents that try to debug them depend on an upstream tool to red

Notation Matters: A Benchmark Study of Token-Optimized Formats in Agentic AI Systems

Model ReleasesDGX agent

arXiv:2605.29676v1 Announce Type: new Abstract: Large language models in Agentic AI systems consume tool schemas and execution results and emit tool invocations as structured data. The default languag

PhAIL: A Real-Robot VLA Benchmark and Distributional Methodology

Model ReleasesDGX agent

arXiv:2605.29710v1 Announce Type: new Abstract: Real-world evaluation of vision-language-action (VLA) policies still rests on binary success rate at a fixed timeout with N le 25 rollouts per condition

PRAIB: Peer Review AI Benchmark of Behaviour of LLM-Assisted Reviewing

Model ReleasesDGX agent

arXiv:2605.29815v1 Announce Type: new Abstract: The growing number of submitted papers has motivated the exploration of Large Language Models (LLMs) as a means to support and augment the peer review p

Recovering Policy-Induced Errors: Benchmarking and Trajectory Synthesis for Robust GUI Agents

SafetyDGX agent

arXiv:2605.29447v1 Announce Type: cross Abstract: While GUI agents have advanced rapidly, they often lack the robustness to recover from their own errors, hindering real-world deployment. To bridge th

Redundant or Necessary? A Benchmark for Detecting Redundant Steps in Agent Trajectories

Model ReleasesDGX agent

arXiv:2605.29893v1 Announce Type: new Abstract: LLM-based agents have demonstrated strong capabilities in solving complex tasks through multi-step reasoning and tool use. However, existing evaluation

SCDBench: A Benchmark for LLM-Based Smart Contract Decompilers

Model ReleasesDGX agent

arXiv:2605.29059v1 Announce Type: cross Abstract: Smart contract decompilation aims to recover high-level source code from bytecode, but evaluating decompilers remains difficult because existing studi

28 May 2026

Ask Now, Use Later: Benchmarking the Proactivity Gap in Long-Lived LLM Agents

Model ReleasesDGX agent

arXiv:2605.28108v1 Announce Type: new Abstract: A long-lived LLM agent, such as OpenClaw, earns its value by acting on a user's preferences and constraints across sessions, not just the current reques

Benchmarking and Mechanistic Analysis of Vision-Language Models for Cross-Depiction Assembly Instruction Alignment

Model ReleasesDGX agent

arXiv:2604.00913v2 Announce Type: replace-cross Abstract: 2D assembly diagrams are often abstract and hard to follow, creating a need for intelligent assistants that can monitor progress, detect error

Benchmarking Ultrasound Foundation Models for Fetal Plane Classification

Model ReleasesDGX agent

arXiv:2605.27796v1 Announce Type: cross Abstract: Ultrasound is widely used in obstetric care due to its safety, accessibility, and real-time imaging. However, interpretation remains operator-dependen

ClinConsensus: A Physician-Calibrated Benchmark for Evaluating Clinical Rubric Coverage in Chinese Medical LLMs

Model ReleasesDGX agent

arXiv:2603.02097v5 Announce Type: replace Abstract: Open-ended medical LLM evaluation remains weakly grounded in physician-calibrated coverage of clinically relevant response criteria, especially in l

DisasterBench: Benchmarking LLM Planning under Typed Tool Interface Constraints

Model ReleasesDGX agent

arXiv:2605.27957v1 Announce Type: new Abstract: Disasters cause severe societal impacts, demanding rapid coordination of heterogeneous AI tools, from satellite analysis to flood prediction and damage

FinBoardBench: Benchmarking Dynamic Wealth Management and Strategic Financial Reasoning of LLMs via Board Game Simulations

ApplicationsDGX agent

arXiv:2605.27896v1 Announce Type: new Abstract: Recently, large language models (LLMs) have achieved superior performance in static financial reasoning and simple dynamic trading tasks. However, exist

High Performance, Low Reliability: Uncertainty Benchmarking for Tabular Foundation Models

Model ReleasesDGX agent

arXiv:2605.28554v1 Announce Type: new Abstract: Recent Tabular Foundation Models (TFMs) have demonstrated state-of-the-art predictive performance, often surpassing Gradient-Boosted Decision Trees (GBD

HRBench: Benchmarking and Understanding Thinking-Mode Switch Strategies in Hybrid-Reasoning LLMs

ResearchDGX agent

arXiv:2605.28398v1 Announce Type: new Abstract: Hybrid-reasoning large language models (LLMs) expose explicit controls over reasoning effort, allowing users or systems to trade off answer quality agai

MIRA: A Bilingual Benchmark for Medical Information Response Audit

Model ReleasesDGX agent

arXiv:2605.28025v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to provide public-facing health information, yet existing safety evaluations overlook whether respons

SuiChat-CN: Benchmarking Contextual Suicide Risk Assessment in Chinese Group Chats

Model ReleasesDGX agent

arXiv:2605.27911v1 Announce Type: new Abstract: Suicide is a critical global public health challenge, causing approximately 720,000 deaths each year and calling for timely, effective prevention strate

What-If World: A Causal Benchmark for General World Models in Embodied Scenarios

Model ReleasesDGX agent

arXiv:2605.27589v1 Announce Type: new Abstract: Video generation models are increasingly used as world simulators for tasks like driving and robotic manipulation. What matters in these settings is not

27 May 2026

Anchor: Mitigating Artifact Drift in Agent Benchmark Generation

Model ReleasesDGX agent

arXiv:2605.26321v1 Announce Type: new Abstract: AI agents are beginning to complete valuable, long-horizon business operations tasks, but training and evaluation environments for enterprise work still

Benchmarking Convolutional, Transformer, Hybrid, and Vision Language Models for Multi Disease Retinal Screening

Model ReleasesDGX agent

arXiv:2605.26283v1 Announce Type: new Abstract: Modern deep learning offers powerful tools for automated retinal screening, but it remains unclear how different visual model families compare in realis

Cesarean Scar Defect Segmentation in Transvaginal Ultrasound Images: a Dataset and Benchmark

Model ReleasesDGX agent

arXiv:2605.26774v1 Announce Type: new Abstract: Cesarean Scar Defect (CSD) is one of the most prevalent complications following cesarean delivery. Transvaginal ultrasonography is widely used for prima

ENPMR-Bench: Benchmarking Proactive Memory Retrieval for Emotional Support Agents

Model ReleasesDGX agent

arXiv:2605.27240v1 Announce Type: new Abstract: Memory-augmented language agents are increasingly deployed in affective applications such as emotional support, where understanding and responding to us

OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modeling

Model ReleasesDGX agent

arXiv:2605.26322v1 Announce Type: new Abstract: Theory of Mind (ToM), the ability to infer others' knowledge, intentions, and emotions, is commonly evaluated in large language models (LLMs) using end-

PersLitEval: Fine-grained Benchmark and Evaluation of LLMs on Persian Literature Questions

Model ReleasesDGX agent

arXiv:2605.27015v1 Announce Type: new Abstract: Despite impressive multilingual capabilities, large language models (LLMs) remain poorly evaluated on literary knowledge in non-English languages. We in

Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History

Model ReleasesDGX agent

arXiv:2602.17003v2 Announce Type: replace-cross Abstract: Large language models have advanced web agents, yet current agents lack personalization capabilities. Since users rarely specify every detail

PRBench: A Standardized Probabilistic Robustness Benchmark

Model ReleasesDGX agent

arXiv:2511.01724v3 Announce Type: replace Abstract: Deep learning models are notoriously vulnerable to imperceptible perturbations. Most existing research centers on adversarial robustness (AR), which

Receipt Replay OOD: A Small Benchmark for Screen Replay Detection Under Domain Shift

Model ReleasesDGX agent

arXiv:2605.26855v1 Announce Type: new Abstract: Public datasets such as DLC-2021, SynID, and KID34K have significantly contributed to research on presentation attack detection for identity documents,

26 May 2026

A Matched Spectral Benchmark of Quantum Inspired Feature Maps

Model ReleasesDGX agent

arXiv:2605.24324v1 Announce Type: cross Abstract: Quantum machine learning is often motivated by the idea that quantum systems can expose useful high-dimensional structure that is difficult to access

AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions

Model ReleasesDGX agent

arXiv:2605.25707v1 Announce Type: new Abstract: Autonomous computer use agents that powered by multimodal large language models (MLLMs) are emerging as capable assistants for completing complex digita

Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs

Model ReleasesDGX agent

arXiv:2605.21602v2 Announce Type: replace Abstract: Many safety and alignment failures of large language models (LLMs) occur due to out-of-distribution (OOD) situations: unusual prompt or response pat

ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions

Model ReleasesDGX agent

arXiv:2605.24279v1 Announce Type: new Abstract: A frontier language model's acknowledged 'helpful programming assistant' persona does not survive long agentic-coding sessions in the deployment regime

FoodMonitor: Benchmarking MLLMs for Explainable Compliance Analysis

Model ReleasesDGX agent

arXiv:2605.24503v1 Announce Type: cross Abstract: As AI-powered compliance monitoring becomes increasingly important in public governance and industrial safety, the ability to provide verifiable evide

LLM-as-a-Reviewer: Benchmarking Their Ability, Divergence, and Prompt Injection Resistance as Paper Reviewers

Model ReleasesDGX agent

arXiv:2605.25415v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used in academic peer review, yet their reliability, alignment with human judgment, and robustness to adve

Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-horizon Agents

Model ReleasesDGX agent

arXiv:2605.25535v1 Announce Type: new Abstract: Existing large language model (LLM) based memory systems apply universal, static policies that overlook a fundamental reality: the contexts that are wor

Rethinking Continual Anomaly Detection on the Edge: Benchmarking Under Realistic Industrial Conditions

Model ReleasesDGX agent

arXiv:2605.24251v1 Announce Type: new Abstract: Continual anomaly detection (CAD) addresses the need for industrial inspection systems to adapt to evolving production conditions, yet existing methods

SkillEvolBench: Benchmarking the Evolution from Episodic Experience to Procedural Skills

Model ReleasesDGX agent

arXiv:2605.24117v1 Announce Type: new Abstract: Large language model (LLM) agents accumulate rich episodic trajectories while solving real-world tasks, but it remains unclear whether such experience c

StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios

Model ReleasesDGX agent

arXiv:2605.25758v1 Announce Type: new Abstract: Large Language Models (LLMs) have reshaped user profiling, yet current evaluations mainly focus on static data snapshots. This paradigm overlooks the re

The Age of Curiosity Meets the Age of AI: Benchmarking Child Safety in Large Language Models

Model ReleasesDGX agent

arXiv:2605.25510v1 Announce Type: new Abstract: Children increasingly have access to Large Language Models (LLMs), which may expose them to responses that are developmentally inappropriate or require

The LSCD Benchmark: a Testbed for Diachronic Word Meaning Tasks

Model ReleasesDGX agent

arXiv:2404.00176v3 Announce Type: replace Abstract: Lexical Semantic Change Detection (LSCD) is a complex, lemma-level task, which is usually operationalized based on two subsequently applied usage-le

TimeSpot: Benchmarking Geo-Temporal Understanding in Vision-Language Models in Real-World Settings

Model ReleasesDGX agent

arXiv:2603.06687v2 Announce Type: replace-cross Abstract: Geo-temporal understanding, the ability to infer location, time, and contextual properties from visual input alone, underpins applications suc

ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks

Model ReleasesDGX agent

arXiv:2605.25388v1 Announce Type: new Abstract: Nucleotide sequences constitute the fundamental genetic basis of biological systems, rendering viral genomic analysis critical for biomedical advancemen

25 May 2026

EvalVerse: Pipeline-Aware and Expert-Calibrated Benchmarking for Professional Cinematic Video Generation

AgentsDGX agent

arXiv:2605.23271v1 Announce Type: cross Abstract: The rapid evolution of generative video foundation models has propelled the field toward professional-grade cinematic synthesis. To achieve such deman

StereoGenBench: A Synthetic Multi-Camera Benchmark for Stereo Generation under Controlled Baseline Regimes

Model ReleasesDGX agent

arXiv:2605.23237v1 Announce Type: new Abstract: Stereo image and video generation, stereo geometry estimation, and condition-controlled view synthesis require paired data in which the variables that d

23 May 2026

Benchmarking Machine Learning Architectures for Antimicrobial Stewardship in Pediatric ICUs

TutorialsDGX agent

arXiv:2605.22611v1 Announce Type: new Abstract: Antimicrobial stewardship (AMS) is critical in pediatric intensive care units (PICUs), where diagnostic uncertainty often drives broad-spectrum antibiot

Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability

Model ReleasesDGX agent

arXiv:2605.22168v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) map complex visual inputs to semantic spaces, but interpreting the cross-modal reasoning of VLMs currently relies on pos

MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents

Model ReleasesDGX agent

arXiv:2602.13372v2 Announce Type: replace-cross Abstract: Evaluating moral alignment in agents navigating conflicting, hierarchically structured human norms is a critical challenge at the intersection

Stabilising Explainability Fragility in Cybersecurity AI: The Impact and Mitigation of Multicollinearity in Public Benchmark Datasets

Model ReleasesDGX agent

arXiv:2605.22529v1 Announce Type: new Abstract: This paper investigates a unexplored yet impactful vulnerability in AI explainability used in intrusion detection (IDS): multicollinearity-induced insta

22 May 2026

AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Grounding

Model ReleasesDGX agent

arXiv:2605.22034v1 Announce Type: new Abstract: Visual grounding, the task of localizing objects described by natural-language expressions, is a foundational capability for agricultural AI systems, en

← Previous
1…6970717273…546
Next →