AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
6 Oct 2026

WasserMan: Benchmark for Underwater Manipulation Policy Learning

Model ReleasesDGX agent

arXiv:2610.04536v1 Announce Type: new Abstract: Underwater manipulation couples visual decisions, contact forces and a thruster-controlled floating base. We introduce WasserMan, to our knowledge the f

Watermarks and Fingerprints as Soft Bindings for Content Provenance: An Open-Licence Benchmark for Images, Audio and Video

Model ReleasesDGX agent

arXiv:2610.04151v1 Announce Type: new Abstract: Content-provenance standards such as C2PA let a platform recover a stripped manifest through a soft binding: an invisible watermark read from the conten

5 Oct 2026

A benchmark for LLMs playing Civilization V. GLM-5.3 is ahead of Opus-5.5, and Qwen-3.8-27B holds up surprisingly well.

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Model ReleasesDGX agent

A while ago, I posted here getting OSS-120B and GLM-4.6 playing full games of Civilization V. Since then, models have moved pretty far, and we wanted a better understanding about models' capabilities

DexJoCo-X: Benchmarking Action Representations for Multi-Hand Dexterous Manipulation

Model ReleasesDGX agent

arXiv:2610.03278v1 Announce Type: new Abstract: As dexterous hands proliferate, collecting data and training policies separately for every morphology becomes increasingly impractical. Scalable cross-e

EditHero: A Benchmark for Long-Horizon Part-Level 3D Editing and Vibe Modeling

Model ReleasesDGX agent

arXiv:2610.02298v1 Announce Type: cross Abstract: 3D editing methods are usually tested on a single edit, yet an asset is built through a long sequence of revisions, each of which must implement the r

Time Series Forecasting Benchmarks Need Scenario-Grounded Stress Testing

ApplicationsDGX agent

arXiv:2610.02608v1 Announce Type: new Abstract: Time series forecasting (TSF) increasingly drives decisions in transportation, energy, finance, healthcare, and infrastructure, yet current evaluation r

2 Oct 2026

Does Continual Imitation Learning Remain Grounded? A Language-Perturbed Benchmark for Robotic Task Retention

Model ReleasesDGX agent

arXiv:2610.00542v1 Announce Type: new Abstract: Continual imitation learning evaluates whether a robot can learn new knowledge without forgetting previously learned skills. However, retaining task per

SimplexUQ: An Evaluation Framework and Benchmark for Conformal Uncertainty on Simplex-Valued Predictions

Model ReleasesDGX agent

arXiv:2610.00523v1 Announce Type: new Abstract: Conformal prediction guarantees marginal coverage, but a single calibration threshold can still spread that coverage unevenly, over-covering easy region

WOMBAT: Whitebox Oracle for Molecular Benchmarking and Attribution Testing

Model ReleasesDGX agent

arXiv:2610.00713v1 Announce Type: new Abstract: When a graph neural network (GNN) explainer produces an unexpected attribution on a molecule, the attribution alone cannot reveal whether the explainer

1 Oct 2026

A 3GPP-Compliant Benchmark Dataset for RIS-Aided Beyond 5G Networks

Model ReleasesDGX agent

arXiv:2609.39058v1 Announce Type: cross Abstract: Reconfigurable Intelligent Surfaces (RIS) are emerging as a key technology for programmable wireless environments in the beyond the fifth generation (

AneumoBench: A Source-Linked Benchmark for Synthetic-Geometry Transfer in Aneurysm CFD

Model ReleasesDGX agent

arXiv:2505.14717v2 Announce Type: replace-cross Abstract: Scientific machine learning uses simulation data to train surrogate models for fast physical-field prediction across geometries. Local shape e

Benchmarking and Enhancing Skill-Level Memory for Partially Observable Robotic Manipulation

Model ReleasesDGX agent

arXiv:2609.38886v1 Announce Type: new Abstract: Recent advances in robot learning have enabled manipulation policies to perform increasingly diverse tasks and generalize across environments. However,

Benchmarking EMlog Calibration for Autonomous Surface Vehicles

SafetyDGX agent

arXiv:2609.39203v1 Announce Type: new Abstract: Accurate velocity measurement is a fundamental requirement for autonomous surface and underwater vehicles. Commonly, velocity is provided by a Doppler v

DDR4/PCIe4 vs DDR5/PCIe5 for LLMs- I benchmarked them for pre-training. What are your thoughts?

Local AiDGX agent

Hello everyone. I've recently ran some experiments comparing DDR4/PCIe4 and DDR5/PCIe5 for AI workstations on a pre-training run, and would like to hear yours thoughts. First of all, and as a summary

MADBench: Benchmarking the Security of Multi-Agent Debate

Model ReleasesDGX agent

arXiv:2609.39146v1 Announce Type: new Abstract: Multi-agent debate (MAD) can improve large language model (LLM) reasoning by allowing multiple agents to exchange and critique their answers to the same

Network-based Spatial Context Retrieval for Open-weight LLMs: A Faithfulness Benchmark for Grounded Geographic Reasoning

Model ReleasesDGX agent

arXiv:2609.39437v1 Announce Type: new Abstract: Large language models (LLMs) encode substantial latent geographic knowledge, yet they reason poorly over space and are unreliable when queried from coor

Talk2Agent: Benchmarking Voice Interfaces for Text Agents

Model ReleasesDGX agent

arXiv:2609.38867v1 Announce Type: new Abstract: Large language model (LLM) computer-use agents are typically evaluated with clean written instructions, despite speech being an increasingly popular int

30 Sept 2026

APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants

ApplicationsDGX agent

arXiv:2609.37559v1 Announce Type: new Abstract: To serve as real-world personal assistants, streaming video models need persistent memory that retains past experiences for later use. Yet existing stre

Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes

Model ReleasesDGX agent

arXiv:2609.36777v1 Announce Type: new Abstract: Frontier coding agents can now write and execute code that authors 3D environments, but whether they reliably understand 3D structure and precisely cont

CoSec: Benchmarking Agent Security in Communities

Model ReleasesDGX agent

arXiv:2609.34790v2 Announce Type: replace-cross Abstract: LLM agents operate in persistent collaborative environments involving multiple users, communities, memories, files, and tools. Community bound

CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning

Model ReleasesDGX agent

arXiv:2609.35823v1 Announce Type: new Abstract: Vision-language models (VLMs) have made substantial progress in autonomous driving, but their success has primarily been studied in ego-centric scenes.

Equally Good, Yet Different: Benchmarking Rashomon sets in AutoML packages

Model ReleasesDGX agent

arXiv:2609.36970v1 Announce Type: new Abstract: The Rashomon effect describes the existence of multiple near-optimal models that achieve comparable performance while offering fundamentally different e

Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RL

Model ReleasesDGX agent

arXiv:2609.38065v1 Announce Type: cross Abstract: Training agents to follow arbitrary instructions is an important goal of multi-task reinforcement learning (RL). Linear temporal logic (LTL) provides

VesselBench-800K: A Large-scale Perception Benchmark for Multimodal Vessel Detection, Counting, and Density Estimation

Model ReleasesDGX agent

arXiv:2609.37003v1 Announce Type: new Abstract: Vessel perception from space is crucial for a wide range of maritime applications, from traffic monitoring to environmental protection. However, most ex

29 Sept 2026

3dgs-sc: a controlled static screen-content benchmark for 3d gaussian splatting

Model ReleasesDGX agent

arXiv:2609.31756v1 Announce Type: new Abstract: Does high image fidelity imply readable screen content in 3D Gaussian Splatting (3DGS)? We introduce 3DGS-SC, a controlled static screen-content dataset

AmbiModBench: Benchmarking Gene Perturbation Prediction Beyond Shared Responses

Model ReleasesDGX agent

arXiv:2609.32527v1 Announce Type: cross Abstract: Predicting cellular responses to genetic perturbations helps prioritize experiments in single-cell genomics, where exhaustive measurement is infeasibl

AnesTRACE: Benchmarking Intraoperative Anesthesia from Multimodal Perception to Multi-step Decision-Making

SafetyDGX agent

arXiv:2609.32740v1 Announce Type: new Abstract: Intraoperative anesthesia requires systems to interpret evolving multimodal evidence, recommend timely management, and revise decisions as patient state

ARCH-B: Architectural Representation, Comprehension and Hierarchy Benchmark

Model ReleasesDGX agent

arXiv:2609.34047v1 Announce Type: new Abstract: Multimodal models increasingly interpret visual environments, but their ability to recognize the same building across photographs, floor plans, elevatio

Beyond Geometry: Benchmarking and Consistency Reasoning for 3D Logical Anomaly Detection

Model ReleasesDGX agent

arXiv:2609.34143v1 Announce Type: new Abstract: Existing 3D industrial anomaly detection mainly targets local geometric deviations. In contrast, many industrial anomalies violate object-level design o

ConflictVLA-Bench: Benchmarking Behavioral Responses of Vision-Language-Action Models to Premise Conflicts

Model ReleasesDGX agent

arXiv:2609.31792v1 Announce Type: cross Abstract: While Vision-Language-Action (VLA) models perform strongly on manipulation tasks, their responses to invalid task premises remain underexplored. Exist

CoSE-E: A Benchmark for Code-switched Speech Evaluation in Enterprise Settings

Model ReleasesDGX agent

arXiv:2609.35645v1 Announce Type: cross Abstract: Code-switching (CS), a seamless alternation between languages within a single utterance, remains a critical challenge in automatic speech recognition

Cross-Dataset Generalization of Bangladeshi Rice Leaf Disease Classifiers: Benchmark, Diagnosis, and Mitigation

Model ReleasesDGX agent

arXiv:2609.31709v1 Announce Type: new Abstract: Cross-dataset transfer in rice leaf disease classification remains a significant challenge, with models trained on one image collection performing subst

Deep Reinforcement Learning for Equity Trading: Benchmarking Actor-Critic Methods with Forward Retraining

Model ReleasesDGX agent

arXiv:2609.31870v1 Announce Type: new Abstract: Consistently profitable trading is difficult because equity markets are noisy, non-stationary, and only partially predictable from historical data. We b

eval-unlearn: Benchmarking unlearning in Text-to-Image Diffusion Models

Model ReleasesDGX agent

arXiv:2609.35269v1 Announce Type: cross Abstract: The rising number of concept unlearning techniques for text-to-image (T2I) diffusion models has produced a fragmented evaluation landscape. Methods ar

Generative AI-Based Data Augmentation for Oral Lesion Classification: The PhotoMOCI Dataset and Benchmark

Model ReleasesDGX agent

arXiv:2609.35226v1 Announce Type: new Abstract: Early detection of oral cancer via photographic imaging presents a promising avenue for large-scale oral cavity screening. However, the development of r

Jev in Medicine: A Benchmark Evaluation. Preliminary Results

Model ReleasesDGX agent

arXiv:2609.34024v1 Announce Type: cross Abstract: Jev is a non-generative 'System One' model that assigns probabilities to predefined answer options and cannot answer outside them. Its accuracy and ca

Large Language Models for Automated Cross-Domain Machine Learning Task Type Identification: A Benchmark Dataset and Evaluation

Model ReleasesDGX agent

arXiv:2609.35335v1 Announce Type: new Abstract: Machine learning task type identification is essential for constructing valid ML pipelines, yet in practice it is typically specified manually. We inves

MemTransfer: Benchmarking Memory Beyond Matched Experience in Embodied Decision-Making

Model ReleasesDGX agent

arXiv:2609.32313v1 Announce Type: new Abstract: Memory lets an embodied agent reuse past experience, yet retaining useful information does not ensure that the agent can apply it when conditions change

NetInjectBench: Benchmarking Indirect Prompt Injection in Tool-Using Large Language Model Agents for Network Operations

Model ReleasesDGX agent

arXiv:2607.10490v2 Announce Type: replace-cross Abstract: Tool-using large language model (LLM) agents are attractive for network operations, but tickets, alerts, logs, runbooks, and ChatOps messages

One-Step Is Optimal: Unconditional Rectified Flows are Noise2Noise Denoisers, and Multi-Step Integration Provably Hurts---A Benchmark and Task-Based Detectability Study on Low-Dose CT

Model ReleasesDGX agent

arXiv:2609.31670v1 Announce Type: new Abstract: Iterative and generative denoisers are increasingly used under the assumption that multi-step refinement outperforms a single regression pass. We show t

OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models

Model ReleasesDGX agent

arXiv:2609.35052v1 Announce Type: new Abstract: Video world models must preserve the visual state of the world over time, but existing evaluation protocols often rely on generated histories, video ref

ORAV: Benchmarking Audio-Video Generation from Multimodal Contexts

Model ReleasesDGX agent

arXiv:2609.34843v1 Announce Type: new Abstract: Audio-video generation using heterogeneous multimodal references has emerged as a new challenge, requiring both compositional control over generation an

SV2V-RSim: A Comprehensive Benchmark for Self-Selective V2V Cooperative Perception with Near-Realistic Data

Model ReleasesDGX agent

arXiv:2609.32863v1 Announce Type: new Abstract: Vehicle-to-Vehicle (V2V) cooperative perception enhances autonomous driving by enabling vehicles to share information beyond their direct line of sight.

THEIA: A Multimodal Dataset and Benchmark for Vision-Language Analysis of Layout

Model ReleasesDGX agent

arXiv:2609.35035v1 Announce Type: new Abstract: The integration of artificial intelligence into computer-aided design frameworks has sparked a shift in the design of analog integrated circuits (ICs),

Traceable Human-to-Humanoid Sign Language Benchmarking

Model ReleasesDGX agent

arXiv:2609.33354v1 Announce Type: new Abstract: Sign data collection is costly, and teleoperation scales poorly, motivating reuse of large video corpora. Humanoid signing requires converting video-der

When Does Geometric View Synthesis Help Wine Label Retrieval? A Public One-Shot Benchmark Across Self-Supervised and Vision-Language Backbones

Model ReleasesDGX agent

arXiv:2609.33359v1 Announce Type: new Abstract: Geometric view synthesis can expand a single wine-label photograph into a training set, but its value with pretrained image encoders is unclear. We stud

Which Self-Improvements Should We Trust? Reliable Self-Improvement When Agents Reuse Their Benchmarks

ResearchDGX agent

arXiv:2609.33180v1 Announce Type: cross Abstract: As recursive self-improvement (RSI) rapidly advances, reliable evaluation becomes critical for guiding adaptive search. RSI typically relies on finite

28 Sept 2026

A Benchmark and Diagnostic Study of Epistemic Admission in Shared Agent Memory

Model ReleasesDGX agent

arXiv:2609.30813v1 Announce Type: new Abstract: Evaluating claim admission in shared agent memory is challenging because repeated claims may be mistaken for independent evidence. An agent may copy or

25 Sept 2026

Artificial Societies Benchmark: A Validation Framework for Synthetic Research

Model ReleasesDGX agent

arXiv:2609.30030v1 Announce Type: new Abstract: A synthetic survey can reproduce the average answer while misrepresenting how people differ, how their answers relate to one another, or how they respon

Benchmarking Arabic--Russian Machine Translation: A Comparison of Fine-tuned NMT and Few-shot LLMs under Rich Morphology and Low Lexical Overlap

Model ReleasesDGX agent

arXiv:2609.29559v1 Announce Type: new Abstract: Arabic-Russian machine translation (MT) remains under-explored due to the rich morphology of Arabic and low lexical overlap between the two languages. W

Benchmarking Argumentative Behaviour of LLMs: A Study of Defences Against Character Attacks

Model ReleasesDGX agent

arXiv:2609.28673v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed as argumentative agents in persuasive dialogues, necessitating rigorous evaluation of their debat

Clinical Intent Extraction: A FHIR-Aligned Representation and the CIRCA Benchmark

Model ReleasesDGX agent

arXiv:2609.29479v1 Announce Type: new Abstract: Prospective clinical actions, the follow-ups, orders, referrals, and instructions that deter-mine what happens to a patient next, are annotated today in

Language Specificity vs. Domain Diversity: Benchmarking Transformers for Bangla Medical NER

Model ReleasesDGX agent

arXiv:2609.29101v1 Announce Type: new Abstract: Medical Named Entity Recognition (NER) for low-resource languages remains a challenging task due to high linguistic variability and a scarcity of domain

24 Sept 2026

Benchmarking Active Spot Selection for Cost-Efficient Spatial Transcriptomics

Model ReleasesDGX agent

arXiv:2609.27208v1 Announce Type: new Abstract: Spatial transcriptomics (ST) measures gene expression in tissue context, but dense capture grids can be costly and may repeatedly sample morphologically

Benchmarking Hyperspectral Foundation Models for Hyperspectral Unmixing

Model ReleasesDGX agent

arXiv:2609.28283v1 Announce Type: new Abstract: Several foundation models dedicated to hyperspectral images have recently been made available. These models are trained on large unlabeled datasets and

Can LLMs Catch a Rigged Backtest? A Clean-Control Calibration Benchmark

Model ReleasesDGX agent

arXiv:2609.28090v1 Announce Type: cross Abstract: Backtest auditing is a calibration problem: high flaw recall is not useful when the model falsely flags matched clean strategies. We build a 96-item p

EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Tasks

SafetyDGX agent

arXiv:2609.28236v1 Announce Type: new Abstract: Long-horizon embodied interaction requires agents to retain and continually update information about the environment as they observe, act, and encounter

EvEMTBench: An Open Benchmark for Machine Learning in Power System Protection

Model ReleasesDGX agent

arXiv:2609.28149v1 Announce Type: cross Abstract: Studies of machine-learning-based power system protection are difficult to compare because task definitions, measurement access, data partitions, metr

Google launches two benchmark-topping speech generation models

Model ReleasesDGX agent

Google LLC today made two new text-to-speech models, Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS, available through its cloud platform. The algorithms have highly similar application programmin

MLPerf Training Introduces Its First LLM Post-Training Benchmark

Model ReleasesDGX agent

MLPerf Training v6.1 adds an agentic reinforcement-learning workload that measures how quickly systems can teach a 397-billion-parameter language model to repair real software projects. The post MLPer

← Previous
1…6162636465…546
Next →