AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
11 May 2026

IntentGrasp: A Comprehensive Benchmark for Intent Understanding

Model ReleasesDGX agent

arXiv:2605.06832v1 Announce Type: cross Abstract: Accurately understanding the intent behind speech, conversation, and writing is crucial to the development of helpful Large Language Model (LLM) assis

LithoBench: Benchmarking Large Multimodal Models for Remote-Sensing Lithology Interpretation

Model ReleasesDGX agent

arXiv:2605.07640v1 Announce Type: cross Abstract: Remote sensing lithology interpretation is fundamental to geological surveys, mineral exploration, and regional geological mapping. Unlike general lan

McNdroid: A Longitudinal Multimodal Benchmark for Robust Drift Detection in Android Malware

Model ReleasesDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2605.06894v1 Announce Type: cross Abstract: Machine learning (ML) in real-world systems must contend with concept drift, adversarial actors, and a spectrum of potential features with varying cos

SEQUOR: A Multi-Turn Benchmark for Realistic Constraint Following

Model ReleasesDGX agent

arXiv:2605.06353v2 Announce Type: replace Abstract: In a conversation, a helpful assistant must reliably follow user directives, even as they refine, modify, or contradict earlier requests. Yet most i

SREGym: A Live Benchmark for AI SRE Agents with High-Fidelity Failure Scenarios

Model ReleasesDGX agent

arXiv:2605.07161v1 Announce Type: new Abstract: AI agents are increasingly used to diagnose and mitigate failures in production systems, known as agentic Site Reliability Engineering (SRE). Current SR

TEA-Bench: A Systematic Benchmarking of Tool-enhanced Emotional Support Dialogue Agent

Model ReleasesDGX agent

arXiv:2601.18700v2 Announce Type: replace Abstract: Emotional Support Conversation requires not only affective expression but also grounded instrumental support to provide trustworthy guidance. Howeve

TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Models

Model ReleasesDGX agent

arXiv:2601.18744v2 Announce Type: replace Abstract: Time series are ubiquitous in real-world scenarios and crucial for applications ranging from energy management to traffic control. Consequently, the

Utility-Preserving De-Identification for Math Tutoring: Investigating Numeric Ambiguity in the MathEd-PII Benchmark Dataset

Model ReleasesDGX agent

arXiv:2602.16571v2 Announce Type: replace Abstract: Large-scale sharing of dialogue data is key to advancing the science of teaching and learning, yet rigorous de-identification remains a major barrie

7 May 2026

Enhancing Agent Safety Judgment: Controlled Benchmark Rewriting and Analogical Reasoning for Deceptive Out-of-Distribution Scenarios

Model ReleasesDGX agent

arXiv:2605.03242v1 Announce Type: new Abstract: Tool-using agent systems powered by large language models (LLMs) are increasingly deployed across web, app, operating-system, and transactional environm

NoisyCausal: A Benchmark for Evaluating Causal Reasoning Under Structured Noise

Model ReleasesDGX agent

arXiv:2605.04313v1 Announce Type: new Abstract: Causal reasoning in natural language requires identifying relevant variables, understanding their interactions, and reasoning about effects and interven

OpenVTON-Bench: A Large-Scale High-Resolution Benchmark for Controllable Virtual Try-On Evaluation

Model ReleasesDGX agent

arXiv:2601.22725v3 Announce Type: replace Abstract: Recent advances in diffusion models have significantly elevated the visual fidelity of Virtual Try-On (VTON) systems, yet reliable evaluation remain

OracleProto: A Reproducible Framework for Benchmarking LLM Native Forecasting via Knowledge Cutoff and Temporal Masking

SafetyDGX agent

arXiv:2605.03762v1 Announce Type: new Abstract: Large language models are moving from static text generators toward real-world decision-support systems, where forecasting is a composite capability tha

ReasonAudio: A Benchmark for Evaluating Reasoning Beyond Matching in Text-Audio Retrieval

Model ReleasesDGX agent

arXiv:2605.03361v2 Announce Type: new Abstract: As multimodal content continues to expand at a rapid pace, audio retrieval has emerged as a key enabling technology for media search, content organizati

UAV as Urban Construction Change Monitor: A New Benchmark and Change Captioning Model

Model ReleasesDGX agent

arXiv:2605.04409v1 Announce Type: new Abstract: Remote Sensing Image Change Captioning (RSICC) aims to generate spatially grounded natural language descriptions of scene evolution from bi-temporal ima

6 May 2026

A Benchmark for Interactive World Models with a Unified Action Generation Framework

Model ReleasesDGX agent

arXiv:2605.03941v1 Announce Type: new Abstract: Achieving Artificial General Intelligence (AGI) requires agents that learn and interact adaptively, with interactive world models providing scalable env

Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarks

Model ReleasesDGX agent

arXiv:2605.03759v1 Announce Type: new Abstract: While Large Vision-Language Models (LVLMs) offer powerful capabilities, they pose privacy risks by unintentionally memorizing sensitive personal informa

Benchmarking Local Language Models for Social Robots using Edge Devices

Local AiDGX agent

arXiv:2605.03111v1 Announce Type: cross Abstract: Social-educational robots designed for socially interactive pedagogical support, such as the Robot Study Companion (RSC), rely on responsive, privacy-

DataClaw: A Process-Oriented Agent Benchmark for Exploratory Real-World Data Analysis

Model ReleasesDGX agent

arXiv:2605.02503v1 Announce Type: new Abstract: Evaluating autonomous data analysis agents requires testing their ability to perform exploratory analysis in underexplored data environments. However, m

EngiBench: A Benchmark for Evaluating Large Language Models on Engineering Problem Solving

Model ReleasesDGX agent

arXiv:2509.17677v2 Announce Type: replace Abstract: Large language models (LLMs) have shown strong performance on mathematical reasoning under well-defined conditions. However, real-world engineering

From Laboratory to Real-World Applications: Benchmarking Agentic Code Reasoning at the Repository Level

Model ReleasesDGX agent

arXiv:2601.03731v3 Announce Type: replace-cross Abstract: As large language models (LLMs) evolve into autonomous agents, evaluating repository-level reasoning, the ability to maintain logical consiste

MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers

Model ReleasesDGX agent

arXiv:2602.00933v2 Announce Type: replace-cross Abstract: The Model Context Protocol (MCP) is rapidly becoming the standard interface for Large Language Models (LLMs) to discover and invoke external t

MHPR: Multidimensional Human Perception and Reasoning Benchmark for Large Vision-Languate Models

Model ReleasesDGX agent

arXiv:2605.03485v1 Announce Type: new Abstract: Multidimensional human understanding is essential for real-world applications such as film analysis and virtual digital humans, yet current LVLM benchma

ReLeaf: Benchmarking Leaf Segmentation across Domains and Species

Model ReleasesDGX agent

arXiv:2605.03784v1 Announce Type: new Abstract: Rising global food demand and growing climate pressure increase the need for sustainable, precise agricultural practices. Automated, individualized plan

5 May 2026

Benchmarking Single-Pose Docking, Consensus Rescoring, and Supervised ML on the LIT-PCBA Library: A Critical Evaluation of DiffDock, AutoDock-GPU, GNINA, and DiffDock-NMDN

Model ReleasesDGX agent

arXiv:2605.01681v1 Announce Type: new Abstract: Virtual screening performance depends heavily on the chosen docking and scoring methods. Recent AI-based tools such as DiffDock and NMDN have reported s

Differentiable Multiphysics Co-Optimization via Implicit Neural Representations: A Transient Hamburger-Cooking Benchmark

Model ReleasesDGX agent

arXiv:2605.01040v1 Announce Type: cross Abstract: The co-optimization of geometry and physical parameters remains challenging in transient multiphysics systems involving moving boundaries, nonlinear m

Extreme Weather Bench: A framework and benchmark for evaluation of high-impact weather

Model ReleasesDGX agent

arXiv:2605.01126v1 Announce Type: new Abstract: Forecasting the wide variety of high-impact weather events experienced globally is a challenge for both Artificial Intelligence (AI) and Numerical Weath

GR-Ben: A General Reasoning Benchmark for Evaluating Process Reward Models

Model ReleasesDGX agent

arXiv:2605.01203v1 Announce Type: cross Abstract: Currently, process reward models (PRMs) have exhibited remarkable potential for test-time scaling. Since large language models (LLMs) regularly genera

LUMINA: A Grid Foundation Model for Benchmarking AC Optimal Power Flow Surrogate Learning

Model ReleasesDGX agent

arXiv:2605.02133v1 Announce Type: new Abstract: AC optimal power flow (ACOPF) is foundational yet computationally expensive in power grid operations, driving learning-based surrogates for large-scale

PepSpecBench: A Unified Evaluation Benchmark for Peptide Tandem Mass Spectrometry Prediction

Model ReleasesDGX agent

arXiv:2605.01945v1 Announce Type: new Abstract: Tandem mass spectrometry provides a high-throughput framework for identifying and quantifying proteins in complex biological samples. In computational p

RMGAP: Benchmarking the Generalization of Reward Models across Diverse Preferences

Model ReleasesDGX agent

arXiv:2605.01831v1 Announce Type: new Abstract: Reinforcement Learning from Human Feedback has become the standard paradigm for language model alignment, where reward models directly determine alignme

Seeing the Scene Matters: Revealing Forgetting in Video Understanding Models with a Scene-Aware Long-Video Benchmark

Model ReleasesDGX agent

arXiv:2603.27259v2 Announce Type: replace Abstract: Long video understanding (LVU) remains a core challenge in multimodal learning. Although recent vision-language models (VLMs) have made notable prog

SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generation

Model ReleasesDGX agent

arXiv:2508.15658v5 Announce Type: replace Abstract: The rapid growth of academic literature makes the manual creation of scientific surveys increasingly infeasible. While large language models show pr

Towards High Fidelity Face Swapping: A Comprehensive Survey and New Benchmark

Model ReleasesDGX agent

arXiv:2605.00883v1 Announce Type: new Abstract: Face swapping has witnessed significant progress in recent years, largely driven by advances in deep generative models such as GANs and diffusion models

4 May 2026

BanglaSocialBench: A Benchmark for Evaluating Sociopragmatic and Cultural Alignment of LLMs in Bangladeshi Social Interaction

Model ReleasesDGX agent

arXiv:2603.15949v3 Announce Type: replace Abstract: Large Language Models have demonstrated strong multilingual fluency, yet fluency alone does not guarantee socially appropriate language use. In high

ControBench: An Interaction-Aware Benchmark for Controversial Discourse Analysis on Social Networks

Model ReleasesDGX agent

arXiv:2605.00513v1 Announce Type: new Abstract: Understanding how people argue across ideological divides online is important for studying political polarization, misinformation, and content moderatio

Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues

ResearchDGX agent

arXiv:2605.00119v1 Announce Type: new Abstract: There is a significant gap in evaluating cultural reasoning in LLMs using conversational datasets that capture culturally rich and dialectal contexts. M

Foresight Arena: An On-Chain Benchmark for Evaluating AI Forecasting Agents

Model ReleasesDGX agent

arXiv:2605.00420v1 Announce Type: cross Abstract: Evaluating the true forecasting ability of AI agents requires environments resistant to overfitting, free from centralized trust, and grounded in ince

1 May 2026

AppTek Call-Center Dialogues: A Multi-Accent Long-Form Benchmark for English ASR

Model ReleasesDGX agent

arXiv:2604.27543v1 Announce Type: new Abstract: Evaluating English ASR systems for conversational AI applications remains difficult, as many publicly available corpora are either pre-segmented into sh

Assessing Pancreatic Ductal Adenocarcinoma Vascular Invasion: the PDACVI Benchmark

Model ReleasesDGX agent

arXiv:2604.27582v1 Announce Type: new Abstract: Surgical resection remains the only potentially curative treatment for pancreatic ductal adenocarcinoma (PDAC), and eligibility depends on accurate asse

Do Papers Tell the Whole Story? A Benchmark and Framework for Uncovering Hidden Implementation Gaps in Bioinformatics

Model ReleasesDGX agent

arXiv:2603.22018v2 Announce Type: replace Abstract: Ensuring consistency between research papers and their corresponding software code implementations is a fundamental prerequisite for guaranteeing th

From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Tasks

ResearchDGX agent

arXiv:2604.27453v1 Announce Type: new Abstract: Large language models have achieved remarkable progress in text generation but still struggle with generative writing tasks. In terms of evaluation, exi

RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension

Model ReleasesDGX agent

arXiv:2601.14289v2 Announce Type: replace-cross Abstract: Understanding research papers remains challenging for foundation models due to specialized scientific discourse and complex figures and tables

30 Apr 2026

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation

Model ReleasesDGX agent

arXiv:2604.26923v1 Announce Type: cross Abstract: LLMs have achieved strong results on both function-level code synthesis and repository-level code modification, yet a capability that falls between th

Graph-based Semantic Calibration Network for Unaligned UAV RGBT Image Semantic Segmentation and A Large-scale Benchmark

Model ReleasesDGX agent

arXiv:2604.26893v1 Announce Type: new Abstract: Fine-grained RGBT image semantic segmentation is crucial for all-weather unmanned aerial vehicle (UAV) scene understanding. However, UAV RGBT semantic s

Human-in-the-Loop Benchmarking of Heterogeneous LLMs for Automated Competency Assessment in Secondary Level Mathematics

Model ReleasesDGX agent

arXiv:2604.26607v1 Announce Type: new Abstract: As Competency-Based Education (CBE) is gaining traction around the world, the shift from marks-based assessment to qualitative competency mapping is a m

LIT-RAGBench: Benchmarking Generator Capabilities of Large Language Models in Retrieval-Augmented Generation

Model ReleasesDGX agent

arXiv:2603.06198v2 Announce Type: replace Abstract: Retrieval-Augmented Generation (RAG) is a framework in which a Generator, such as a Large Language Model (LLM), produces answers by retrieving docum

29 Apr 2026

Benchmarking and Improving GUI Agents in High-Dynamic Environments

Model ReleasesDGX agent

arXiv:2604.25380v1 Announce Type: new Abstract: Recent advancements in Graphical User Interface (GUI) agents have predominantly focused on training paradigms like supervised fine-tuning (SFT) and rein

Benchmarking OCR Pipelines with Adaptive Enhancement for Multi-Domain Retail Bill Digitization

Model ReleasesDGX agent

arXiv:2604.25176v1 Announce Type: new Abstract: The digitization of multi-domain retail billing documents remains a challenging task due to variability in scan quality, layout heterogeneity, and domai

DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams

Model ReleasesDGX agent

arXiv:2604.25231v1 Announce Type: cross Abstract: Diagram question answering (DQA) requires models to interpret structured visual representations such as charts, maps, infographics, circuit schematics

DV-World: Benchmarking Data Visualization Agents in Real-World Scenarios

Model ReleasesDGX agent

arXiv:2604.25914v1 Announce Type: new Abstract: Real-world data visualization (DV) requires native environmental grounding, cross-platform evolution, and proactive intent alignment. Yet, existing benc

EOS-Bench: A Comprehensive Benchmark for Earth Observation Satellite Scheduling

Model ReleasesDGX agent

arXiv:2604.25782v1 Announce Type: cross Abstract: Earth observation satellite imaging scheduling is a challenging NP-hard combinatorial optimisation problem central to space mission operations. While

PSP: An Interpretable Per-Dimension Accent Benchmark for Indic Text-to-Speech

Model ReleasesDGX agent

arXiv:2604.25476v1 Announce Type: cross Abstract: Standard text-to-speech (TTS) evaluation measures intelligibility (WER, CER) and overall naturalness (MOS, UTMOS) but does not quantify accent. A synt

28 Apr 2026

Benchmarking and Mitigating Sycophancy in Medical Vision Language Models

Model ReleasesDGX agent

arXiv:2509.21979v4 Announce Type: replace-cross Abstract: Visual language models (VLMs) have the potential to transform medical workflows. However, the deployment is limited by sycophancy. Despite thi

Benchmarking Pathology Foundation Models for Breast Cancer Survival Prediction

Model ReleasesDGX agent

arXiv:2604.24679v1 Announce Type: new Abstract: Pathology foundation models (PFMs) have recently emerged as powerful pretrained encoders for computational pathology, enabling transfer learning across

CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics

Model ReleasesDGX agent

arXiv:2509.20374v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated strong performance across general NLP tasks, but their utility in automating numerical experime

CorpusQA: A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoning

Model ReleasesDGX agent

arXiv:2601.14952v2 Announce Type: replace-cross Abstract: While large language models now handle million-token contexts, their capacity for reasoning across entire document repositories remains largel

CT-FineBench: A Diagnostic Fidelity Benchmark for Fine-Grained Evaluation of CT Report Generation

Model ReleasesDGX agent

arXiv:2604.24001v1 Announce Type: new Abstract: The evaluation of generated reports remains a critical challenge in Computed Tomography (CT) report generation, due to the large volume of text, the div

DO-Bench: An Attributable Benchmark for Diagnosing Object Hallucination in Vision-Language Models

Model ReleasesDGX agent

arXiv:2604.22822v1 Announce Type: cross Abstract: Object level hallucination remains a central reliability challenge for vision language models (VLMs), particularly in binary object existence verifica

Event-based SLAM Benchmark for High-Speed Maneuvers

Model ReleasesDGX agent

arXiv:2604.24033v1 Announce Type: new Abstract: Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution,

SIV-Bench: A Video Benchmark for Social Interaction Understanding and Reasoning

Model ReleasesDGX agent

arXiv:2506.05425v2 Announce Type: replace-cross Abstract: Understanding social interaction, which encompasses perceiving numerous and subtle multimodal cues, inferring unobservable mental states and r

← Previous
1…5859606162…546
Next →