AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
22 May 2026

ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoning

Model ReleasesDGX agent

arXiv:2605.22734v1 Announce Type: new Abstract: Biomedical knowledge graphs (KGs) treat disease associations as static facts, but temporal information is crucial for clinical reasoning, e.g., a sympto

Security Document Classification with a Fine-Tuned Local Large Language Model: Benchmark Data and an Open-Source System

Model ReleasesDGX agent

arXiv:2605.20368v1 Announce Type: cross Abstract: Organizations that scan documents for sensitive information face a practical problem. Cloud services require data to be sent to external infrastructur

STRUCTSENSE: A Task-Agnostic Agentic Framework for Structured Information Extraction with Human-In-The-Loop Evaluation and Benchmarking

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Agents
DGX agent

arXiv:2507.03674v3 Announce Type: replace Abstract: Extracting structured information from scientific literature is critical for accelerating discovery, yet Large Language Models (LLMs) often struggle

21 May 2026

ArchSIBench: Benchmarking the Architectural Spatial Intelligence of Vision-Language Models

Model ReleasesDGX agent

arXiv:2605.20837v1 Announce Type: new Abstract: Architectural spatial intelligence, the ability to recognize and infer architectural space, is fundamental to tasks such as robot navigation, embodied i

Benchmarking Empirical and Learning-Based Approaches for Feedforward Steering Control in Autonomous Racing

Model ReleasesDGX agent

arXiv:2605.21111v1 Announce Type: new Abstract: Feedforward steering control is a key component of hierarchical control architectures for autonomous racing. The goal is to reduce steering corrections

Fine-grained Claim-level RAG Benchmark for Law

Model ReleasesDGX agent

arXiv:2605.21071v1 Announce Type: new Abstract: The rapid progress of large language models (LLMs) is shifting semantic search toward a question-answering paradigm, where users ask questions and LLMs

GraphRAG on Consumer Hardware: Benchmarking Local LLMs for Healthcare EHR Schema Retrieval

Model ReleasesDGX agent

arXiv:2605.20815v1 Announce Type: new Abstract: Graph-based Retrieval Augmented Generation (GraphRAG) extends retrieval-augmented generation to support structured reasoning over complex corpora, but i

HalluCXR: Benchmarking and Mitigating Hallucinations in Medical Vision-Language Models for Chest Radiograph Interpretation

Model ReleasesDGX agent

arXiv:2605.20469v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly used for medical image interpretation, yet they frequently hallucinate, generating clinically plausible b

Lowering the Barrier to IREX Participation: Open-Source Algorithms, Toolkit, and Benchmarking for Iris Recognition

ResearchDGX agent

arXiv:2605.20735v1 Announce Type: new Abstract: This paper proposes two new open-source iris recognition algorithms, providing both Python and IREX-compliant C++ implementations to be submitted to the

ShadeBench: A Benchmark Dataset for Building Shade Simulation in Sustainable Society

Model ReleasesDGX agent

arXiv:2605.20510v1 Announce Type: new Abstract: Urban heat exposure is becoming an increasingly critical challenge due to the intensifying urban heat island effect. Fine-grained shade patterns, especi

20 May 2026

Benchmarking and Evolving Reason-Reflect-Rectify for Reflective Visual Generation

Model ReleasesDGX agent

arXiv:2605.19639v1 Announce Type: new Abstract: Text-to-Image (T2I) models and Unified Multimodal Models (UMMs) have achieved remarkable progress in visual generation. However, their reliance on a sin

BuildArena: A Physics-Aligned Interactive Benchmark of LLMs for Engineering Construction

Model ReleasesDGX agent

arXiv:2510.16559v5 Announce Type: replace Abstract: Engineering construction automation aims to transform natural language specifications into physically viable structures, requiring complex integrate

CogScale: Scalable Benchmark for Sequence Processing

Model ReleasesDGX agent

arXiv:2605.19758v1 Announce Type: new Abstract: The ability to maintain and manipulate information over time is a fundamental aspect of living beings and Artificial Intelligence. While modern models h

CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing

Model ReleasesDGX agent

arXiv:2605.19484v1 Announce Type: cross Abstract: While GUI agents have made significant progress in web navigation and basic operating system tasks, their capabilities in professional creative workfl

Eyes on VLM: Benchmarking Gaze Following and Social Gaze Prediction in Vision Language Models

ResearchDGX agent

arXiv:2605.19859v1 Announce Type: new Abstract: Vision-language models (VLMs) have rapidly evolved into general-purpose multimodal reasoners with strong zero-shot generalization. In this context, VLMs

MixRea: Benchmarking Explicit-Implicit Reasoning in Large Language Models

Model ReleasesDGX agent

arXiv:2605.20128v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly integrated into high-stakes decision-making. Inspired by the theory of inattentional blindness in human co

POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agents

Model ReleasesDGX agent

arXiv:2605.19127v1 Announce Type: new Abstract: LLM agents increasingly have access to private user data and act on the user's behalf when interacting with third-party systems. The user defines what m

The Evaluation Game: Beyond Static LLM Benchmarking

Model ReleasesDGX agent

arXiv:2605.19377v1 Announce Type: cross Abstract: As jailbreaks, adversarially crafted inputs that bypass safety constraints, continue to be discovered in Large Language Models, practitioners increasi

XNote: Benchmarking Automated Community Notes Generation for Image-based Contextual Deception

Model ReleasesDGX agent

arXiv:2603.22453v2 Announce Type: replace Abstract: Community Notes have emerged as an effective crowd-sourced mechanism for combating online deception on social media platforms. However, its reliance

19 May 2026

A Comprehensive Survey of Action Quality Assessment: Method and Benchmark

Model ReleasesDGX agent

arXiv:2412.11149v2 Announce Type: replace Abstract: Action Quality Assessment (AQA) aims to automatically evaluate how well human actions are performed and has been widely applied in sports analysis,

A2RBench: An Automatic Paradigm for Formally Verifiable Abstract Reasoning Benchmark Generation

Model ReleasesDGX agent

arXiv:2605.17278v1 Announce Type: new Abstract: Abstract reasoning ability reflects the intelligence and generalization capacity of LLMs to extract and apply abstract rules. However, accurately measur

AgroCoT: A Chain-of-Thought Benchmark for Evaluating Reasoning in Vision-Language Models for Agriculture

Model ReleasesDGX agent

arXiv:2511.23253v3 Announce Type: replace Abstract: Recent advancements in Vision-Language Models (VLMs) have significantly impacted various industries. In agriculture, these multimodal capabilities h

Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models

Model ReleasesDGX agent

arXiv:2605.18504v1 Announce Type: new Abstract: Machine Translation (MT) for Ancient Greek (AG) to Modern Greek (MG) is a low-resource task, constrained by the lack of large-scale, high-quality parall

BlendedNet++: A dataset and benchmark for field-resolved aerodynamics and inverse design of blended wing body aircraft

Model ReleasesDGX agent

arXiv:2512.03280v2 Announce Type: replace-cross Abstract: The conceptual design of Blended Wing Body (BWB) aircraft is often constrained by the high computational cost of resolving complex aerodynamic

CANSURF: An ASV-View Can Dataset and Benchmark for Detection and Tracking of Surface-Level Debris

Model ReleasesDGX agent

arXiv:2605.16774v1 Announce Type: cross Abstract: Surface-level marine debris remains a practical bottleneck for autonomous clean-up, where small, reflective targets (e.g., aluminum cans) must be dete

Causely: A Causal Intelligence Layer for Enterprise AI A Benchmark Study on SRE and Reliability Workflows

Model ReleasesDGX agent

arXiv:2605.18327v1 Announce Type: new Abstract: AI agents deployed into SRE workflows currently derive their understanding of environment state from raw observability telemetry at query time, paying a

CrossView Suite: Harnessing Cross-view Spatial Intelligence of MLLMs with Dataset, Model and Benchmark

Model ReleasesDGX agent

arXiv:2605.18621v1 Announce Type: cross Abstract: Spatial intelligence requires multimodal large language models (MLLMs) to move beyond single-view perception and reason consistently about objects, vi

DBES: A Systematic Benchmark and Metric Suite for Evaluating Expert Specialization in Large-Scale MoEs

Model ReleasesDGX agent

arXiv:2605.18498v1 Announce Type: cross Abstract: Expert specialization in Mixture-of-Experts (MoE) models remains poorly understood, with traditional evaluations conflating architectural load-balanci

iMiGUE-3K: A Large-Scale Benchmark for Micro-Gesture Analysis with Self-Supervised Learning

Model ReleasesDGX agent

arXiv:2605.17179v1 Announce Type: new Abstract: Emotion understanding is a fundamental challenge in affective computing and artificial intelligence. While existing approaches predominantly focus on fa

LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injectio

Model ReleasesDGX agent

arXiv:2605.17986v1 Announce Type: cross Abstract: AI agents such as OpenClaw are increasingly deployed in local workflows with access to external tools. This creates indirect prompt-injection (IPI) ri

PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments

Model ReleasesDGX agent

arXiv:2603.23231v2 Announce Type: replace Abstract: Empowering large language models with long-term memory is crucial for building agents that adapt to users' evolving needs. Existing evaluations of t

SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors

Model ReleasesDGX agent

arXiv:2605.16626v1 Announce Type: cross Abstract: Since autonomous coding agents generate complex behaviors at high-volume, we may want to use other LLMs to monitor actions to reduce the risk from dan

Time Series Foundation Models as Strong Baselines in Transportation Forecasting: A Large-Scale Benchmark Analysis

Model ReleasesDGX agent

arXiv:2602.24238v2 Announce Type: replace Abstract: Accurate forecasting of transportation dynamics is essential for urban mobility and infrastructure planning. Although recent work has achieved stron

TriALS: Triphasic-Aided Liver Lesion Segmentation Benchmark in Non-Contrast CT

Model ReleasesDGX agent

arXiv:2605.16572v1 Announce Type: new Abstract: Automated segmentation of liver lesions on non-contrast computed tomography (NCCT) is clinically important but fundamentally challenging, particularly i

Validate Your Authority: Benchmarking LLMs on Multi-Label Precedent Treatment Classification

Model ReleasesDGX agent

arXiv:2605.17691v1 Announce Type: cross Abstract: Automating the classification of negative treatment in legal precedent is a critical yet nuanced NLP task where misclassification carries significant

18 May 2026

AGOP-IxG: A Gradient Covariance Filter for Local Feature Attribution on Tabular Data, with a Controlled Benchmark

Model ReleasesDGX agent

arXiv:2605.15700v1 Announce Type: new Abstract: Automated machine learning pipelines increasingly produce models whose predictions must be explained to end users, auditors, and downstream decision sys

IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia

Model ReleasesDGX agent

arXiv:2603.17915v2 Announce Type: replace-cross Abstract: As large language models (LLMs) are deployed in multilingual settings, their safety behavior in culturally diverse, low-resource languages rem

Mind the Motions: Benchmarking Theory-of-Mind in Everyday Body Language

ResearchDGX agent

arXiv:2511.15887v2 Announce Type: replace Abstract: Our ability to interpret others' mental states through nonverbal cues (NVCs) is fundamental to our survival and social cohesion. While existing Theo

MyoChallenge 2025: A New Benchmark for Human Athletic Intelligence

Model ReleasesDGX agent

arXiv:2605.15650v1 Announce Type: new Abstract: Athletic performance represents the pinnacle of human motor intelligence, demanding rapid choices, precise control, agility, and coordinated physical ex

VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing

Model ReleasesDGX agent

arXiv:2605.15677v1 Announce Type: new Abstract: Despite the rapid advancements in Vision-Language Models (VLMs), a critical gap remains in their ability to handle structured, controllable diagrammatic

15 May 2026

Enhancing Few-Shot Classification of Benchmark and Disaster Imagery with ABHFA-Net

Model ReleasesDGX agent

arXiv:2510.18326v3 Announce Type: replace Abstract: The rising incidence of natural and human-induced disasters necessitates robust visual recognition systems capable of operating under limited labele

NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces

Model ReleasesDGX agent

arXiv:2605.14698v1 Announce Type: cross Abstract: Foundation models (FMs) promise to extract unified representations that generalize across downstream tasks. They have emerged across fields, including

OPT-Engine: Benchmarking the Limits of LLMs in Optimization Modeling via Complexity Scaling

Model ReleasesDGX agent

arXiv:2601.19924v2 Announce Type: replace-cross Abstract: We investigate the capabilities and scalability of Large Language Models (LLMs) in optimization modeling, a domain requiring structured reason

PROCESS-2: A Benchmark Speech Corpus for Early Cognitive Impairment Detection

Model ReleasesDGX agent

arXiv:2605.14888v1 Announce Type: cross Abstract: Speech-based analysis offers a scalable and non-invasive approach for detecting cognitive decline, yet progress has been constrained by the limited av

Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuning

Model ReleasesDGX agent

arXiv:2605.13936v1 Announce Type: cross Abstract: The recent success of large language models (LLMs) has been largely driven by vast public datasets. However, the next frontier for LLM development lie

ViMU: Benchmarking Video Metaphorical Understanding

Model ReleasesDGX agent

arXiv:2605.14607v1 Announce Type: new Abstract: Any new medium, once it emerges, is used for more than the transmission of overt content alone. The information it carries typically operates on two lev

14 May 2026

AIvaluateXR: An Evaluation Framework for on-Device AI in XR with Benchmarking Results

Local AiDGX agent

arXiv:2502.15761v3 Announce Type: replace-cross Abstract: The deployment of large language models (LLMs) on extended reality (XR) devices has great potential to advance the field of human-AI interacti

Benchmarking Attribute Discrimination in Infant-Scale Vision-Language Models

Model ReleasesDGX agent

arXiv:2512.18951v3 Announce Type: replace Abstract: Infants learn not only object categories but also fine-grained visual attributes such as color, size, and texture from limited experience. Prior inf

CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence

Model ReleasesDGX agent

arXiv:2605.12882v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have significantly advanced document understanding, yet current Doc-VQA evaluations score only the final answ

SpikeProphecy: A Large-Scale Benchmark for Autoregressive Neural Population Forecasting

Model ReleasesDGX agent

arXiv:2605.12992v1 Announce Type: cross Abstract: Neural population models, which predict the joint firing of many simultaneously recorded neurons forward in time, are typically evaluated by a single

Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?

Model ReleasesDGX agent

arXiv:2605.12684v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) are now routinely deployed for visual understanding, generation, and curation. A substantial fraction of thes

13 May 2026

Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters

Model ReleasesDGX agent

arXiv:2605.11960v1 Announce Type: new Abstract: Vision Large Language Models (VLLMs) have achieved remarkable success in modern text-rich visual understanding. However, their perceptual robustness in

MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces

HardwareDGX agent

arXiv:2605.11333v1 Announce Type: cross Abstract: The fast pace of artificial intelligence~(AI) innovation demands an agile methodology for observation, reproduction and optimization of distributed ma

SafeManip: A Property-Driven Benchmark for Temporal Safety Evaluation in Robotic Manipulation

Model ReleasesDGX agent

arXiv:2605.12386v1 Announce Type: new Abstract: Robotic manipulation is typically evaluated by task success, but successful completion does not guarantee safe execution. Many safety failures are tempo

12 May 2026

A Stability Benchmark of Generative Regularizers for Inverse Problems

Model ReleasesDGX agent

arXiv:2605.10076v1 Announce Type: cross Abstract: Generative (diffusion) priors demonstrate remarkable performance in addressing inverse problems in imaging. Yet, for scientific and medical imaging, i

AgentRx: A Benchmark Study of LLM Agents for Multimodal Clinical Prediction Tasks

Model ReleasesDGX agent

arXiv:2605.10286v1 Announce Type: new Abstract: Building effective clinical decision support systems requires the synthesis of complex heterogeneous multimodal data. Such modalities include temporal e

BenchHAR: Benchmarking Self-Supervised Learning for Generalizable Sensor-based Activity Recognition

Model ReleasesDGX agent

arXiv:2605.08296v1 Announce Type: new Abstract: Human Activity Recognition (HAR) from wearable sensors supports broad healthcare and behavior science applications. However, data heterogeneity and the

Benchmarking Compositional Generalisation for Machine Learning Interatomic Potentials

Model ReleasesDGX agent

arXiv:2605.08988v1 Announce Type: cross Abstract: Machine Learning Interatomic Potentials play a fundamental role in computational chemistry and materials science, enabling applications from molecular

Benchmarking Sensor-Fault Robustness in Forecasting

Model ReleasesDGX agent

arXiv:2605.10822v1 Announce Type: new Abstract: Cyber-physical system (CPS) forecasting models depend on sensor streams with noisy, biased, missing, or temporally misaligned readings, yet standard for

Beyond Isolation: A Unified Benchmark for General-Purpose Navigation

Model ReleasesDGX agent

arXiv:2605.09441v1 Announce Type: new Abstract: The pursuit of general-purpose embodied agents is hindered by fragmented evaluation protocols that isolate navigation skills and fixate on specific robo

← Previous
1…7071727374…546
Next →