AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
25 Sept 2026

GeoRefer-Bench: A Benchmark from Referring Pixels to Verifiable Geospatial Reasoning

Model ReleasesDGX agent

arXiv:2609.29541v1 Announce Type: cross Abstract: Referring segmentation in overhead imagery is inherently relational: a query may ask for the buildings north of the road or the pond closest to a resi

IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents

Model ReleasesDGX agent

arXiv:2601.06676v3 Announce Type: replace-cross Abstract: Large Language Model (LLM)-based deep research agents perform multi-step reasoning, web exploration, and long-form report generation. In these

Limited Structural Reliability in Public Educational Prediction Benchmarks: A Four-Dimension Audit of Seven Datasets

Model ReleasesDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2609.29625v1 Announce Type: new Abstract: Across seven public educational prediction datasets, three passed all four pre-modeling reliability checks; the remaining four either failed group-aware

OptiSAR-Net++: A Large-Scale Benchmark and Transformer-Free Framework for Cross-Domain Remote Sensing Visual Grounding

Model ReleasesDGX agent

arXiv:2603.24876v2 Announce Type: replace Abstract: Remote sensing visual grounding (RSVG) aims to localize specific targets in remote sensing images using natural language expressions. However, exist

PFArena: Benchmarking Language Models for Protein Modification

Model ReleasesDGX agent

arXiv:2609.28921v1 Announce Type: new Abstract: Protein modification requires navigating an immense sequence space, yet wet-lab validation remains low-throughput and costly. Although computational par

STRAND: Benchmarking and Improving Object-Centric Spatio-Temporal Monitoring in Video Large Language Models

Model ReleasesDGX agent

arXiv:2609.29607v1 Announce Type: cross Abstract: While multimodal large language models (MLLMs) have advanced video understanding, they remain highly prone to hallucinations in dynamic scenes. We arg

24 Sept 2026

A Systematic Benchmark of Explainable Methods for Temporal Attribution in Sequential Recommendation Systems

Model ReleasesDGX agent

arXiv:2609.27201v1 Announce Type: new Abstract: Sequential RecSys are central to modern personalization, exploiting user's historical interaction sequences to drive next-step decisions. Deep learning

Beyond Balanced Accuracy: A Resolution and Parity-Controlled Benchmark for Vision-Language and Vision-Only Defect Assessment in UAV Power-Line Inspection

Model ReleasesDGX agent

arXiv:2609.27457v1 Announce Type: new Abstract: Vision-language models (VLMs) are often reported to outperform task-specific vision backbones for unmanned aerial vehicle (UAV) power-line defect assess

Does Graph Structure Earn Its Place in Microservice Root-Cause Analysis? A Controlled Study on RCAEval, and What the Benchmark Was Really Measuring

Model ReleasesDGX agent

arXiv:2609.27069v1 Announce Type: cross Abstract: Graph neural networks dominate recent work on microservice root-cause analysis, yet recent results question whether the graph contributes. Those resul

Phonemizing User-Generated Text: A Benchmark, Taxonomy, and Compositional Approach

Model ReleasesDGX agent

arXiv:2609.27205v1 Announce Type: cross Abstract: Text-to-speech systems increasingly process user-generated text (UGT) such as ppl and imo, whose pronunciation must be inferred from the canonical rat

WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents

AgentsDGX agent

arXiv:2609.27490v1 Announce Type: new Abstract: AI research agents need reliable knowledge of how their experiments change outcomes. We introduce WhatWorkedBench to measure experimental understanding,

23 Sept 2026

FinFIRST: Benchmarking Search Agents for Financial Information Retrieval, Sourcing and Traceability

Model ReleasesDGX agent

arXiv:2609.25192v1 Announce Type: new Abstract: Financial search is a highly demanding task for LLM agents, requiring not only a correct final answer but also temporally valid information retrieval, a

FinInteract: Benchmarking Clarification and Intent Integration in Ambiguous Financial Question Answering

Model ReleasesDGX agent

arXiv:2609.24002v1 Announce Type: new Abstract: Large language model agents increasingly answer financial questions by searching regulatory filings. Such questions are often deceptively under-specifie

FireWorldBench: Benchmarking Complex Physical World Intelligence through Coupled-Field Fire Dynamics

Model ReleasesDGX agent

arXiv:2609.23064v1 Announce Type: new Abstract: Understanding the physical world requires more than object recognition, scene description, and short-term visual prediction, as real-world physical syst

MCPGen: Benchmarking LLMs on Executable MCPWorkflow Development

Model ReleasesDGX agent

arXiv:2609.23925v1 Announce Type: cross Abstract: We study whether LLMs can produce executable workflow artifacts that remain consistent across graph structure, tool implementation, schema bindings, a

PhysAI-Bench: A Benchmark for LLM-Based Agentic Decision-Making in Autonomous UAV-Centric Physical AI

Model ReleasesDGX agent

arXiv:2609.23695v1 Announce Type: new Abstract: Recent advances in Physical AI have accelerated the use of foundation models in autonomous systems such as unmanned aerial vehicles (UAVs), which must p

SpecialEduBench: Benchmarking Vision-Language Models on Knowledge, Skill, and Attitude in Language Intervention for Autistic Children

Model ReleasesDGX agent

arXiv:2609.26090v1 Announce Type: new Abstract: Language is the target of most early intervention for autistic children. Because the goal and the method change from child to child, the work falls to a

TicTacBench: Benchmarking Timing Closure Capabilities of Coding Agents

Model ReleasesDGX agent

arXiv:2609.23363v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have led to the emergence of coding agents capable of performing complex engineering tasks, including re

TimeLitmus: A Diagnostic Benchmark for Cross-Modal Understanding and Explanation Faithfulness in Event-Conditioned Time-Series Prediction

Model ReleasesDGX agent

arXiv:2609.24677v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to make predictions from numerical time-series histories and textual events. Yet accuracy alone canno

22 Sept 2026

BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models

Model ReleasesDGX agent

arXiv:2511.18921v2 Announce Type: replace Abstract: Backdoor attacks undermine the reliability and trustworthiness of machine learning systems by injecting hidden behaviors that can be maliciously act

CCTU: A Benchmark for Tool Use under Complex Constraints

Model ReleasesDGX agent

arXiv:2603.15309v2 Announce Type: replace Abstract: Solving problems through tool use under explicit constraints constitutes a highly challenging yet unavoidable scenario for large language models (LL

CytoCrowd: A Multi-Annotator Benchmark Dataset for Cytology Image Analysis

Model ReleasesDGX agent

arXiv:2602.06674v2 Announce Type: replace Abstract: High-quality annotated datasets are crucial for advancing machine learning in medical image analysis. However, a critical gap exists: most datasets

H2RBench: A Real-to-Sim Benchmark for Evaluating Human-to-Robot Transfer

Model ReleasesDGX agent

arXiv:2609.24778v1 Announce Type: new Abstract: Learning robot manipulation policies from human video demonstrations constitutes a promising avenue for scalable robot learning. However, comparing diff

MemCalib: Benchmarking and Optimizing Memory Use in LLM Agents

Model ReleasesDGX agent

arXiv:2609.24259v1 Announce Type: new Abstract: The effectiveness of agent memory ultimately depends on whether the underlying LLM gives each memory in context an appropriate degree of influence over

MinCU: A Fine-Grained Benchmark for Grounded Minimal-Change Understanding in Image Pairs

Model ReleasesDGX agent

arXiv:2609.23336v1 Announce Type: new Abstract: Localizing and describing fine-grained differences between near-identical images is a critical yet underexplored capability for multimodal large languag

Mind the Gaps: A Curated Benchmark for Form Field Detection

Model ReleasesDGX agent

arXiv:2609.23679v1 Announce Type: new Abstract: Form Field Detection (FFD) is a fundamental component of document understanding systems, enabling applications ranging from large-scale industrial digit

Omni2Web: Benchmarking Audiovisual Website Development

Model ReleasesDGX agent

arXiv:2609.23417v1 Announce Type: cross Abstract: Screen-recorded web editing requests contain weak deictic expressions such as ``this'' and ``there,'' whose referents depend on speech, cursor traject

Rethinking Class Imbalance for Single-Cell Foundation Models: A Systematic Benchmark Across Architectures and Long-Tail Loss Functions

Model ReleasesDGX agent

arXiv:2609.23325v1 Announce Type: new Abstract: Single-cell foundation models (scGPT, scBERT, Geneformer) achieve cell-type classification accuracy up to 97.5% in our experiments, yet this aggregate a

When Is Availability-Aware Training Worth It? A Benchmark and Empirical Study of Interruption-Resilient Optimization Under Predictable Compute Schedules

Model ReleasesDGX agent

arXiv:2609.22087v1 Announce Type: cross Abstract: Training under non-stationary but predictable compute availability (satellites under eclipse, duty-cycled edge devices, power-capped datacenters) is o

21 Sept 2026

Benchmarking Autonomous Driving Planners Across Leaderboards: A Unified CARLA-Based Evaluation

Model ReleasesDGX agent

arXiv:2509.22754v2 Announce Type: replace-cross Abstract: Autonomous driving remains a highly active research domain that seeks to enable vehicles to perceive dynamic environments, predict the future

Chinese Competitive Debating Dataset and Benchmark

Model ReleasesDGX agent

arXiv:2609.21637v1 Announce Type: cross Abstract: Debate adjudication requires tracking how arguments develop through interaction, yet existing datasets rarely combine fine-grained debate transcripts

DPed-VLN: A Benchmark for Socially Compliant Vision-and-Language Navigation in Dynamic Pedestrian Environments

Model ReleasesDGX agent

arXiv:2609.21504v1 Announce Type: new Abstract: Vision-and-language navigation (VLN) has advanced rapidly in static indoor environments, but robots operating in human-populated spaces must ground lang

MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs

Model ReleasesDGX agent

arXiv:2609.20850v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) show remarkable advancements, their cross-modal capabilities introduce complex vulnerabilities that easil

OpenMAS-GCom. A Diagnostic Benchmark for Graph-enhanced Multi-Agent Systems

Model ReleasesDGX agent

arXiv:2609.21527v1 Announce Type: new Abstract: Graph-enhanced multi-agent systems (G-MAS) coordinate large language model agents through communication graphs and role assignments, which determine how

PlaceReasoner-Beta: Reasoning-Driven Macro Placement and Benchmarking

Model ReleasesDGX agent

arXiv:2609.21263v1 Announce Type: new Abstract: Automated macro placement remains a fundamental challenge in VLSI physical design. Despite decades of research, existing approaches predominantly optimi

PRISM-BN: A Controlled Corpus and Benchmark for Text-to-Parameterized Bayesian Network Extraction

Model ReleasesDGX agent

arXiv:2609.21673v1 Announce Type: new Abstract: Probabilistic Graphical Models (PGMs), especially Bayesian Networks (BNs), expose directed structure and probabilistic parameters, making them natural s

VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration

Model ReleasesDGX agent

arXiv:2609.21521v1 Announce Type: cross Abstract: While Video Large Language Models (Video-LLMs) have recently demonstrated strong performance, reliably evaluating their fine-grained video understandi

18 Sept 2026

BioPhys-Bridge: A Benchmark for Interdisciplinary Scientific Reasoning in Physics-Grounded Biological Research

Model ReleasesDGX agent

arXiv:2609.19180v1 Announce Type: new Abstract: Language models face unique challenges in analyzing interdisciplinary scientific research literature. In biophysics research, faithful answers require g

BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software

Model ReleasesDGX agent

arXiv:2509.25248v2 Announce Type: replace-cross Abstract: Automatically compiling open-source software (OSS) projects is a vital, labor-intensive, and complex task, which makes it a good challenge for

17 Sept 2026

Benchmarking Visual-Inertial Odometry in Subterranean Environments Under Sensor Degradation, Miscalibration, and Dynamic Occlusion

Model ReleasesDGX agent

arXiv:2609.18628v1 Announce Type: new Abstract: Visual-inertial odometry (VIO) is a core capability for autonomous operation in GPS-denied subterranean environments, yet its reliability can degrade sh

iMINDBench: iEEG Multi-Institution Neural Decoding Benchmark

Model ReleasesDGX agent

arXiv:2609.18104v1 Announce Type: new Abstract: Intracranial electroencephalography (iEEG) is widely used to record electrical activity directly from electrodes inside the human brain, making it an at

16 Sept 2026

BRAVE-6D: Benchmark for Robotic Active Vision in 6DOF Pose Estimation

Model ReleasesDGX agent

arXiv:2609.17106v1 Announce Type: new Abstract: Detecting and grasping small objects remains a significant challenge in robotics. Active vision, where the robot moves closer to the object, is an intui

What Breaks Local Watermarks? A Robustness Benchmark for Local Invisible Image Watermarking

Model ReleasesDGX agent

arXiv:2609.16832v1 Announce Type: cross Abstract: Local image watermarking embeds an invisible signal into selected image regions rather than spreading it across the entire image, enabling payload rec

15 Sept 2026

Bangla Sentence Function Classification: Corpus Development, Model Benchmarking, and Interpretability

Model ReleasesDGX agent

arXiv:2609.13869v1 Announce Type: cross Abstract: Automatic sentence function identification is important for many downstream natural language processing (NLP) applications such as dialogue systems, t

Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration

Model ReleasesDGX agent

arXiv:2609.15669v1 Announce Type: cross Abstract: Multimodal image registration is a key component of many clinical workflows, yet it remains challenging because corresponding anatomical structures of

Benchmarking Optimizers to Solve Inverse Problems with Differentiable Physics Simulators

Model ReleasesDGX agent

arXiv:2609.13819v1 Announce Type: new Abstract: Solving inverse problems with differentiable physics simulators holds the potential to revolutionize scientific discovery and engineering design, as it

EMCStereo: Attention-Enhanced Stereo Matching for Thin-Structure Depth Estimation with a Synthetic Tree-Branch Benchmark

Model ReleasesDGX agent

arXiv:2609.13233v1 Announce Type: new Abstract: Thin structures such as tree branches are among the hardest cases for stereo matching: a branch is only a few pixels wide, the background is cluttered,

GP-VMimesSMA: Benchmarking General-Purpose Vision Models and Specialized Architectures for 2D Medical Image Segmentation

ResearchDGX agent

arXiv:2603.13044v2 Announce Type: replace-cross Abstract: Medical image segmentation (MIS) is a fundamental component of computer-assisted diagnosis and clinical decision support. Over the past decade

K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations

Model ReleasesDGX agent

arXiv:2609.15855v1 Announce Type: cross Abstract: % !TEX root = ../main.tex People increasingly use large language models (LLMs) for mental health support, yet their safety in evolving, high-risk conv

SomBench: Benchmark Dataset for Advancing Machine Learning in Lunar Science

Model ReleasesDGX agent

arXiv:2609.13277v1 Announce Type: new Abstract: Lunar orbital missions, such as Lunar Reconnaissance Orbiter, Kaguya/SELENE, Gravity Recovery and Interior Laboratory, and Lunar Prospector, among other

The MODA General Attribute Suite: A Four-Track Evaluation Benchmark for Fashion Attribute Extraction

Model ReleasesDGX agent

arXiv:2609.13279v1 Announce Type: new Abstract: Fashion attribute extraction is evaluated inconsistently: results are reported as single aggregate numbers across image types that pose different proble

Toward Secure Code Generation: Bridging Correctness and Security via Task-Adaptive Vulnerability Modeling and Execution-Based Benchmarking

Model ReleasesDGX agent

arXiv:2407.02395v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for program synthesis, yet they often generate code that is functionally plausible but inse

14 Sept 2026

Statistical Uncertainty Quantification for Aggregate Performance Metrics in Machine Learning Benchmarks

Model ReleasesDGX agent

arXiv:2501.04234v2 Announce Type: replace-cross Abstract: Modern artificial intelligence is supported by machine learning models (e.g., foundation models) that are pretrained on a massive data corpus

UrduFactCheck: An Agentic Fact-Checking Framework for Urdu with Evidence Boosting and Benchmarking

AgentsDGX agent

arXiv:2505.15063v3 Announce Type: replace Abstract: The rapid adoption of Large Language Models (LLMs) has raised important concerns about the factual reliability of their outputs, particularly in low

11 Sept 2026

Beyond Benchmarks: Using VLMs to Reveal Systematic Classification Failures Under Real World Conditions

ApplicationsDGX agent

arXiv:2609.11126v1 Announce Type: new Abstract: Verification and validation (V&V) of classification models is crucial to enable a wide range of sensor processing applications. Currently, the V&V proce

CMNIE: An Information Extraction Benchmark for Chinese Military News

Model ReleasesDGX agent

arXiv:2609.10722v1 Announce Type: new Abstract: Structured extraction from Chinese military news supports intelligence analysis, decision-making, and knowledge base construction. However, existing res

ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs

Model ReleasesDGX agent

arXiv:2609.10895v1 Announce Type: new Abstract: Reacting to sudden physical hazards (catching a slipping plate, dodging a falling knife) is both a meaningful test of embodied intelligence and a hard r

10 Sept 2026

Beyond Cross-Lingual Transfer: Benchmarking Propagation Boundaries in Multilingual LLM Unlearning

Model ReleasesDGX agent

arXiv:2609.05976v1 Announce Type: cross Abstract: Large Language Model (LLM) unlearning aims to suppress target knowledge while preserving general capabilities. In multilingual settings, unlearning mu

LogiScope-VQA: Benchmarking Vision-Language Models for Logistics Hazard Identification in Industrial Scenarios

Model ReleasesDGX agent

arXiv:2609.09790v1 Announce Type: cross Abstract: Large Multimodal Models (LMMs) large-scale deployment in industrial warehouse settings specifically necessitates that models exhibit human-expert-leve

YallaMorph: A Benchmark for Evaluating Arabic Morphological Generation in Large Language Models

Model ReleasesDGX agent

arXiv:2609.10153v1 Announce Type: new Abstract: Arabic morphology remains challenging for large language models, since fluent generation does not guarantee accurate morphosyntactic control. Existing A

← Previous
1…4849505152…546
Next →