AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
8 Oct 2026

RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments

ResearchDGX agent

arXiv:2610.10409v1 Announce Type: cross Abstract: General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities carr

7 Oct 2026

Wiki-Talkie: Multilingual Benchmarking of Persona-Based Agents on Real-World Discussions

AgentsDGX agent

arXiv:2610.08513v1 Announce Type: cross Abstract: LLMs are increasingly deployed as autonomous agents in social environments, making it critical to study their ability to faithfully simulate human int

6 Oct 2026

Benchmarking CLIP for Zero-Shot Face and Periocular Gender Estimation

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
SafetyDGX agent

arXiv:2610.06102v1 Announce Type: new Abstract: We investigate CLIP for zero-shot gender estimation from full-face and periocular images. Three CLIP backbones are evaluated on 11,299 frontal images fr

4 Oct 2026

Here are some pictures of a robot costume wearing high-specularity edge-case mirror suit, a dataset (425 RAW/JPEGs) for benchmarking CV & depth-estimation algorithms against extreme mirror reflections [D]

ApplicationsDGX agent

Purpose-built to stress-test computer vision models, depth cameras, and spatial AI against severe specular glare and geometric reflections. This 425-asset production archive features a custom faceted

Nonobench: an open benchmark of 49 LLMs on nonogram puzzles, public and open source [P]

Model ReleasesDGX agent

Nonobench measures how well LLMs solve nonograms (picross). Each model gets the row and column clues once and returns the full grid. No tools, one attempt per puzzle. Method: - Standard mode: 30 puzzl

2 Oct 2026

VERITYGATE: A Four-Gate Schema-Level Faithfulness Framework and Paired Benchmark for Grounded LLM Narrations over Structured Evidence

Model ReleasesDGX agent

arXiv:2610.00833v1 Announce Type: cross Abstract: Fluent LLM explanations may not follow the evidence from a structured system. We present VERITYGATE, a four-gate checker for declared evidence IDs, en

1 Oct 2026

From Benchmarks to Production: Transferring Time Series Anomaly Detection Methods for Electricity Production Monitoring

ApplicationsDGX agent

arXiv:2609.39257v1 Announce Type: cross Abstract: Accurate forecasting of electricity production is essential for maintaining the operational efficiency and strategic planning of energy utilities. In

Hyperspectral Image Dataset for Benchmarking on Salient Object Detection

ResearchDGX agent

arXiv:1806.11314v3 Announce Type: replace Abstract: Many works have been done on salient object detection using supervised or unsupervised approaches on colour images. Recently, a few studies demonstr

NAQD Env: A benchmark for selective withdrawal in language agents

Model ReleasesDGX agent

arXiv:2609.38460v1 Announce Type: new Abstract: Language agents must revise planned actions when evidence changes, permission is revoked, or a stop instruction arrives. A useful response is selective:

30 Sept 2026

AnthroDial: Benchmarking LLM Anthropomorphism in Autonomous Social Interaction

AgentsDGX agent

arXiv:2609.37853v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as social agents, yet credible human-like interaction requires more than fluent responses or pers

Evaluating and Benchmarking the System One Model Jev

Model ReleasesDGX agent

arXiv:2609.37647v1 Announce Type: cross Abstract: Jev is a commercial System One model from TypeSafe AI that does not generate text: given a state and typed questions, it returns a choice from fixed o

Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method

Model ReleasesDGX agent

arXiv:2609.35965v1 Announce Type: cross Abstract: Vision-and-Language Navigation (VLN) has largely focused on a single agent following a single instruction, yet many real-world applications require te

29 Sept 2026

Language-Augmented Video Action Anticipation: Design Fundamentals, Benchmarks, and Open Challenges

ApplicationsDGX agent

arXiv:2609.31665v1 Announce Type: new Abstract: Action anticipation predicts future human actions from partial video under incomplete context and temporal uncertainty. Recent systems introduce large l

25 Sept 2026

BanglaTurn: A Benchmark and Whisper-Based Model for End-of-Turn Detection in Bangla Speech

Model ReleasesDGX agent

arXiv:2609.29371v1 Announce Type: new Abstract: This paper presents BanglaTurn, a corpus for end-of-turn detection in Bangla conversational speech, and a model trained on it. The corpus holds 35,374 s

fable.intermittent: benchmarking probabilistic forecasting methods for intermittent time series

Model ReleasesDGX agent

arXiv:2609.28607v1 Announce Type: new Abstract: Intermittent time series are common in spare-parts demand and retail sales. Since the cost of forecast errors is typically asymmetric, decisions such as

I benchmarked tool-call reliability on qwen2.5-coder against pi, Hermes, and Goose - all three got 0%

Local AiDGX agent

A while back I posted here about a CLI agent I'm building that parses tool calls out of a model's plain-text output instead of trusting native function-calling to work. Finally ran the actual numbers.

PHOSA: Photorealistic 3D Sign Avatar Modeling and Benchmark

Model ReleasesDGX agent

arXiv:2609.29292v1 Announce Type: new Abstract: In this work, we focus on photorealistic sign avatar modeling, which is crucial for effective communication with the Deaf community and is characterized

When Identical Rows Disagree: From Benchmark Identifiability to Replication-Robust Anomaly Detection

Model ReleasesDGX agent

arXiv:2609.29580v1 Announce Type: new Abstract: A released table is often treated as an i.i.d. sample, although its repeated rows may encode business frequency, repeated entities, joins, resampling, o

24 Sept 2026

ThinkingCap 3.8-27B vs. Swift 3.8-27B vs. Qwen 3.8-27B Benchmarks

Model ReleasesDGX agent

With the release of ThinkingCap-Qwen3.8-27B, I thought it would be worthwhile to do a comparison between the original Qwen3.8-27B, the new ThinkingCap, and Swift-Qwen3.8-27B. Both Swift which I alread

23 Sept 2026

Compared to What? A Human-Anchored Security Benchmark for LLM-Generated Infrastructure-as-Code

Model ReleasesDGX agent

arXiv:2608.28021v2 Announce Type: replace-cross Abstract: Large language models increasingly author Infrastructure-as-Code (IaC), where one insecure default is provisioned straight into production. Pr

Deep Generative Crystal Structure Prediction: A Benchmark Study and a Controlled Test of Prototype Dependence

Model ReleasesDGX agent

arXiv:2609.26502v1 Announce Type: cross Abstract: Deep generative models are widely reported to enable de novo crystal structure prediction (CSP), but their capability has not been measured consistent

Radiomics--Foundation Fusion for Interpretable RCC Classification: Internal Benchmarking and Exploratory External Transfer

ResearchDGX agent

arXiv:2609.26578v1 Announce Type: new Abstract: Accurate preoperative subtype classification of renal cell carcinoma (RCC) from contrast-enhanced CT remains clinically challenging because clear cell R

22 Sept 2026

Benchmarking Hybrid Deep Learning Architectures for Predictive Maintenance in Industry 4.0

TutorialsDGX agent

arXiv:2609.22583v1 Announce Type: new Abstract: Predictive maintenance in Industry 4.0 refers to using data from sensors, machines, and production systems to estimate when equipment is likely to fail,

Benchmarking Off-the-Shelf Multimodal AI Models Against Dermatologists on Patient-Captured Skin Images

Model ReleasesDGX agent

arXiv:2609.24190v1 Announce Type: new Abstract: Artificial intelligence (AI) has advanced at a rapid pace in recent years. Initially, breakthroughs in large language models caught widespread attention

MuLA-Bench: A Multilingual Long-Form Audio Understanding Benchmark via Multi-Tier Auditing

Model ReleasesDGX agent

arXiv:2609.23416v1 Announce Type: cross Abstract: Long-form audio performance is often summarized by context length and aggregate accuracy, obscuring how language, evidence, and task jointly shape dif

Opus 5.5 made me curse at Claude less, which is an unfortunate benchmark to now be using but a win.

Model ReleasesDGX agent

On 22 September 2026, Allie K. Miller posted on Twitter that the Opus 5.5 model led her to curse at Claude less frequently. She notes that while reducing profanity toward another AI is an undesirable

Vision Transformers versus convolutional neural networks for fine-grained orchid genus identification in a species-rich, data-poor flora: a controlled benchmark on the Orchidaceae of New Guinea

Model ReleasesDGX agent

arXiv:2609.24064v1 Announce Type: new Abstract: New Guinea is the world's richest island flora (~2,856 orchid species), yet most species are represented by only a handful of photographs, far fewer tha

21 Sept 2026

Benchmarking Gender Bias in Machine Translation Evaluation Metrics across Occupations

SafetyDGX agent

arXiv:2609.21490v1 Announce Type: new Abstract: Gender bias remains a persistent concern in machine translation (MT), affecting both generated translations and their automatic evaluation. When a sourc

Beyond Kinematics: Benchmarking Simulation Fidelity for Muscle-Driven Imitation Learning

SafetyDGX agent

arXiv:2609.21909v1 Announce Type: new Abstract: In this work, we conduct a systematic comparison of two state-of-the-art motion-imitation reinforcement learning (MIRL) pipelines, one built on SCONE/Hy

18 Sept 2026

Before the Arrest: Benchmarking LLMs on Criminal Profiling from Incomplete Evidence

ApplicationsDGX agent

arXiv:2609.19965v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly applied to legal and criminal justice tasks, yet existing work focuses almost exclusively on post-arrest s

Is It Still Worth Training a Classical Model in the Era of LLMs? A Crossover Benchmark on Tabular Data

Model ReleasesDGX agent

arXiv:2609.20218v1 Announce Type: cross Abstract: Large language models can label a tabular row from a plain-English description with no training - a capability now shipping in mainstream spreadsheet

17 Sept 2026

Benchmarking Tabular Foundation Models as Surrogates in Expensive Evolutionary Optimization

ResearchDGX agent

arXiv:2609.18130v1 Announce Type: cross Abstract: Surrogate-assisted evolutionary algorithms (SAEAs) are effective methods for solving expensive optimization problems (EOPs), where surrogate models re

Prosthesis-Aware 3D Human Pose Estimation: A Dataset and Benchmark for RSP Users

Model ReleasesDGX agent

arXiv:2609.18406v1 Announce Type: new Abstract: Recovering 3D human body motion from video is important for applications such as rehabilitation assessment and sports performance evaluation. For prosth

16 Sept 2026

A Sentinel-2 benchmark dataset for deep-learning active-fire segmentation across 25 California wildfires

Model ReleasesDGX agent

arXiv:2609.16199v1 Announce Type: cross Abstract: This article describes an open image dataset for developing and evaluating active-fire segmentation methods in satellite imagery. The dataset contains

Benchmarking Factual Robustness of LLMs via Multi-conversation Persuasion

SafetyDGX agent

arXiv:2609.16777v1 Announce Type: new Abstract: As Large Language Models (LLMs) increasingly serve as primary knowledge retrieval interfaces, their robustness against extit{persuasion attacks}---attem

Lesion-centered 3D mapping of colonoscopy procedures: validation of a hierarchical ensemble pipeline on public benchmark videos

Model ReleasesDGX agent

arXiv:2609.16672v1 Announce Type: new Abstract: Background and Objective: Colonoscopy recording practice preserves text reports and still photographs, while the spatial information already present in

14 Sept 2026

Duplicating baseline benchmarks [D]

ResearchDGX agent

Suppose I create two machine learning models suppose tree and neural network for a task let's suppose regression problem, now suppose I am sending both of this paper to two different journals, now the

MAxBench: A Multinomial Concept Recovery Benchmark

Model ReleasesDGX agent

arXiv:2609.13072v1 Announce Type: cross Abstract: Fine-grained control of language model behaviors (e.g., steering) is among the more actionable outcomes of interpretability research. For binary conce

11 Sept 2026

How Much Velocity Does Off-Ball Space Value Need? A Broadcast-Viewport Benchmark

Model ReleasesDGX agent

arXiv:2609.10801v1 Announce Type: new Abstract: Velocity-aware pitch control is standard, but under a broadcast viewport half the players are off screen and on-screen velocities come from a drifting c

8 Sept 2026

I built a thing that benchmarks Ollama models on your own hardware and ranks them

Model ReleasesDGX agent

I kept picking local models badly. Parameter count, quantization, VRAM. I could read all of it and still not know whether a model would be any good on my machine. So I stopped guessing and wrote somet

7 Sept 2026

New Benchmark: The Struggle Bench

Model ReleasesDGX agent

How it works. The model being tested is given a server capable of running it's weights and full context. That server is placed in a median priced apartment. The AI is given a bank account with for ren

6 Sept 2026

4xRadeon AI Pro R9700 people, how are your benchmarks?

Model ReleasesDGX agent

Was considering choosing a 4xRadeon AI Pro R9700 build for a office AI server. Curious to know what results you guys are getting? Mainly considering DeepSeek V4 flash and Qwen 3.8 Flash with offloadin

4 Sept 2026

Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks

ResearchDGX agent

arXiv:2609.03734v1 Announce Type: cross Abstract: BLEU-4 is the standard metric for evaluating sign language translation (SLT), but spoken-language metrics may not adequately reflect sign language pro

1 Sept 2026

MMPCBench: Benchmarking Multimodal Large Language Models on Proactive Critique of Flawed Inputs

SafetyDGX agent

arXiv:2608.29286v1 Announce Type: new Abstract: As Multimodal Large Language Models (MLLMs) evolve into sophisticated interactive assistants, their reliability depends not only on following instructio

31 Aug 2026

Benchmarking large language model agent societies against human behavioural distributions

HardwareDGX agent

arXiv:2608.28182v1 Announce Type: cross Abstract: Populations of large language model agents are increasingly used as experimental societies. Three doubts shadow every such result: whether the agents

28 Aug 2026

Classical and Hybrid Quantum Machine Learning for Trigger-Like Event Selection on CMS Open Data: An Eight-Qubit, PCA-Constrained Benchmark

Model ReleasesDGX agent

arXiv:2608.26224v1 Announce Type: cross Abstract: Event triggering sits at the heart of high-energy physics, where the rare events of interest must be retained while an overwhelming background is disc

27 Aug 2026

Fusing Perceptual Vision Experts with Multimodal Large Language Models for Explainable Plant Disease Diagnosis: From Benchmark Imagery to Real-World Robotic Field Validation

Model ReleasesDGX agent

arXiv:2608.24934v1 Announce Type: new Abstract: Accurate field plant disease diagnosis requires reliable fusion of uncertain and conflicting perceptual evidence. We present the Hybrid Hierarchical Mul

26 Aug 2026

Benchmarking local agent architecture (ReAct loop vs. Graph)

Local AiDGX agent

Hi all! I've been building a local agent harness, but before writing the code, I wanted to get some intuition around whether (and how) structuring agent execution as a graph helps vs. standard tool-ca

Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility

ResearchDGX agent

arXiv:2601.17027v2 Announce Type: replace-cross Abstract: While synthetic data has proven effective for improving scientific reasoning in the text domain, multimodal reasoning remains constrained by t

TrustShiftProbe: Characterizing, Benchmarking, and Defending Staged Trust Attacks on MCP Servers

AgentsDGX agent

arXiv:2608.23763v1 Announce Type: cross Abstract: The Model Context Protocol (MCP) has emerged as the standard layer connecting Large Language Model agents to external tool backends. This openness int

25 Aug 2026

Benchmarking Retrieval-Augmented Generation Strategies for Large Language Model-Based Travel Mode Choice Prediction

ResearchDGX agent

arXiv:2508.17527v2 Announce Type: replace Abstract: Accurately predicting travel mode choice is essential for effective transportation planning, yet traditional statistical and machine learning models

Definitional Sensitivity in Media Bias Detection: A Multi-Definition Dataset and Benchmark

Model ReleasesDGX agent

arXiv:2608.23095v1 Announce Type: new Abstract: Media bias detection relies on definitions and examples that specify what counts as bias, yet these specifications often vary across datasets or remain

KAN-Robust-Bench: A Benchmark for Evaluating the Robustness of Kolmogorov-Arnold Networks

Model ReleasesDGX agent

arXiv:2608.21488v1 Announce Type: cross Abstract: While machine learning models have demonstrated strong performance in many domains, these models have shown profound vulnerabilities when they are exp

No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios

SafetyDGX agent

arXiv:2608.21775v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in real-world applications, yet they remain vulnerable to generating harmful content. From advers

Noise Floor Audit for Agent Benchmarks

Model ReleasesDGX agent

arXiv:2608.22331v1 Announce Type: new Abstract: We audit measurement variability for 3 native tool-calling endpoints across 2 providers on the official BFCL multiple and parallel categories, using mat

24 Aug 2026

[Benchmark] llama.cpp batch/ubatch impacts on PP and TG

Model ReleasesDGX agent

My test is running DeepSeek v4 Flash 0731 at native size on DGX Spark machine (GB10, 128 GB unified memory). The model size is bigger than RAM, so weights will be loaded many times when running. To im

Multilingual Verifier Bias in RLVR: Benchmark, Rollout Diagnosis, and the Cross-Lingual Selection Bottleneck

Model ReleasesDGX agent

arXiv:2608.20362v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) is a standard recipe for training large language models on mathematical reasoning, where an answer

OmniAssistBench: Assistant-style Interaction Benchmark for Omni-LLMs

Model ReleasesDGX agent

arXiv:2608.21360v1 Announce Type: new Abstract: Recent omni-modal large language models (Omni-LLMs) show great potential as real-time video assistants, which continuously perceive environments and gui

22 Aug 2026

Benchmarking qwen3.8:27b by having it port a 34k line C game to one HTML file. The agent mattered more than I expected.

Model ReleasesDGX agent

My hobby project skill-issue is an arena FPS in a single C file, about 34k lines, everything procedural. My test for local setups is to have an agent port it to a single file HTML version with three.j

20 Aug 2026

Been tweaking my Qwen 3.8 setup, up to 45+ steady T/ps at 8bit quant. Realised I'm now top T/ps for this model+ctx across all benchmarked M-series chips. Full args linked below, happy to discuss as this was a pain of trial and error.

Model ReleasesDGX agent

https://omlx.ai/benchmarks/performance/2pko3m1k - you can expand the raw args, but I have full annotations of what worked and what didn't. I'm now testing the model on acutal coding and haven't seen a

← Previous
1…7475767778…546
Next →