AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
9 Sept 2026

AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories

Model ReleasesDGX agent

arXiv:2609.06972v1 Announce Type: cross Abstract: LLM agents complete tasks by issuing sequences of tool calls, and every observation they read is a channel through which an indirect prompt injection

EvoNav-Bench: Benchmarking Lifelong Navigation in Evolving Environments

Model ReleasesDGX agent

arXiv:2609.08292v1 Announce Type: new Abstract: Lifelong navigation (LN) requires an embodied agent to solve a sequence of navigation subtasks in the same environment. Since solving each subtask from

HealthLoopQA: A Context-Aware Question Answering Benchmark for Interpreting Wearable Monitoring Data in Diabetes Care

Model ReleasesDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2609.06976v1 Announce Type: new Abstract: As medical wearables become integrated into daily chronic disease care, effectively interpreting longitudinal monitoring data is essential for patients

LatentMD: Benchmarking Markdown Boundary Failures in LLM-Generated Text

Model ReleasesDGX agent

arXiv:2609.06993v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly generate Markdown that is consumed by renderers, agents, code extractors, and structured downstream pipeline

MEMOBench: A Process Level Memory Benchmark for Robotic Manipulation

Model ReleasesDGX agent

arXiv:2609.07047v1 Announce Type: new Abstract: Robotic manipulation often requires acting on information that is no longer visible, yet Vision-Language-Action policies are usually evaluated when the

TabBench-Bio: A Living Benchmark for Machine Learning on High-Dimensional Biomedical Tables

Model ReleasesDGX agent

arXiv:2609.07441v1 Announce Type: new Abstract: Biomedical tables often combine thousands of measured variables with only tens or hundreds of labelled samples, a regime that is poorly represented in g

7 Sept 2026

AxQM: A Textbook-Scale Benchmark for Formal Proof Synthesis in a Library of Finite-Dimensional Quantum Mechanics

Model ReleasesDGX agent

arXiv:2609.05157v1 Announce Type: cross Abstract: Formalizing mathematics in a proof assistant, where a machine checks every definition, statement and proof, has set a new standard of rigor. Large lan

Corporate-Family Resolution Is Not a String-Matching Problem: A Public Benchmark Stratified by Name Visibility

Model ReleasesDGX agent

arXiv:2609.04269v1 Announce Type: cross Abstract: Deciding whether two supplier records belong to the same corporate family is a prerequisite for spend consolidation, credit exposure aggregation and s

MMTClinic: Multimodal, Multilingual Time Series Question Answering and Reasoning Benchmark for Clinical Domain

Model ReleasesDGX agent

arXiv:2609.04842v1 Announce Type: cross Abstract: Time-series data in clinical settings is crucial for capturing dynamic changes in a patient's health over time, enabling timely diagnosis, personalize

MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Model

Model ReleasesDGX agent

arXiv:2603.18892v2 Announce Type: replace-cross Abstract: Spatial reasoning is foundational for Vision-Language Models (VLMs), particularly when deployed as Vision-Language-Action (VLA) agents in phys

NavArena: Automated Construction of Goal-Oriented Navigation Benchmarks from 3D Gaussian Splatting Reconstructions

Model ReleasesDGX agent

arXiv:2609.04602v1 Announce Type: new Abstract: Fixed 3D Gaussian Splatting (3DGS) reconstructions provide realistic novel views but lack the traversability constraints, valid goals, and closed-loop p

SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding

Model ReleasesDGX agent

arXiv:2609.05141v1 Announce Type: new Abstract: Scientific papers require models to reason jointly over text, equations, figures, tables, code, and datasets while preserving the provenance of supporti

TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agents

Model ReleasesDGX agent

arXiv:2609.05079v1 Announce Type: new Abstract: Autonomous coding agents are increasingly proposed as AI-scientist systems that conduct analyses and write research reports, but executing a prescribed

4 Sept 2026

FrameBench:A Language Understanding Benchmark Based on Frame Semantics

Model ReleasesDGX agent

arXiv:2609.03370v1 Announce Type: new Abstract: In frame semantics, sentence comprehension is assumed to proceed by relating lexical meaning to background knowledge called semantic frames, thereby ena

ThousandWorlds: A benchmark for climate emulation of potentially habitable exoplanets

Model ReleasesDGX agent

arXiv:2606.18338v2 Announce Type: replace Abstract: The search for life beyond Earth will depend on detecting faint signatures in the atmospheres of potentially habitable exoplanets. Interpreting thos

Updated my benchmark with a new vLLM based recipe for Qwen 3.8 Flash Next : now up to 98/100 (instead of 91 previously)

Model ReleasesDGX agent

I was using: weights https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4 with the optimized SGLANG (patched) from https://old.reddit.com/r/BlackwellPerformance/comments/1w04xb7/qwen38_flashnext_o

3 Sept 2026

A Physics-Consistent Benchmark for Contact-Rich Human-Robot Interaction in Assistive Care

Model ReleasesDGX agent

arXiv:2609.02402v1 Announce Type: new Abstract: Conventional task-level evaluation asks whether a robot policy completes a specified action, but can miss failures that emerge only during physical huma

GeoStore: Finding Small Storefronts in Large Scenes -- A Fine-Grained POI Localization Benchmark with Global-to-Local Asymmetric Matching

Model ReleasesDGX agent

arXiv:2609.02012v1 Announce Type: new Abstract: Point-of-interest (POI) localization -- matching a user's close-up storefront photograph against large-scale geo-tagged street-view imagery -- underpins

Sim2Signal: Sim-to-Real Benchmarks for Traffic Signal Control

Model ReleasesDGX agent

arXiv:2609.01676v1 Announce Type: new Abstract: Reinforcement learning achieves strong traffic signal control performance in simulation, yet policies trained in simulators often fail once deployed in

T2LSC-Bench: Benchmarking Localized Semantic Control in Text-to-Image Generation

Model ReleasesDGX agent

arXiv:2609.02255v1 Announce Type: new Abstract: Recent text-to-image models have become increasingly capable of rendering explicit text, but reliable localized text control requires more than generati

2 Sept 2026

My Minimax H3 Workflow Benchmark Data -

Model ReleasesDGX agent

Alright, I posted that I had my agent test a bunch of different workflows for over 12 hours and got the 'Bro just wasted 12 hours of credits'. It was obvious the proof should come from the visual data

1 Sept 2026

A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving

Model ReleasesDGX agent

arXiv:2608.29005v1 Announce Type: new Abstract: Camera-only end-to-end (E2E) driving models are nearing deployment, where the camera stream is degraded by blur, noise, low light, weather, frame loss,

CORAL: A Benchmark for Structure-aware and Brain-wide Neuron Reconstruction in Light Microscopy

Model ReleasesDGX agent

arXiv:2608.30768v1 Announce Type: new Abstract: Automatic neuron reconstruction from light microscopy images is a central problem in computational neuroanatomy. While recent methods have achieved enco

LIBERO-Para: A Diagnostic Benchmark and Metrics for Paraphrase Robustness in VLA Models

Model ReleasesDGX agent

arXiv:2603.28301v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models achieve strong performance in robotic manipulation by leveraging pre-trained vision-language backbones. However,

MMLongEmbed: Benchmarking Multimodal Embedding Models in Long-Context Scenarios

Model ReleasesDGX agent

arXiv:2606.14747v2 Announce Type: replace-cross Abstract: Recent advancements have significantly expanded the theoretical context windows of Multimodal Embedding Models (MEMs). However, larger context

Towards AI-Assisted Research Writing: Benchmarking LLMs for AI/ML Introduction Generation

Model ReleasesDGX agent

arXiv:2508.14273v3 Announce Type: replace Abstract: As researchers increasingly adopt LLMs as writing assistants, generating high-quality research paper introductions remains both challenging and esse

Towards Stream Learning on Embedded Systems: Benchmarking the Memory Consumption of Stream Learning Methods

Model ReleasesDGX agent

arXiv:2608.30923v1 Announce Type: cross Abstract: Stream learning is commonly evaluated through predictive performance and adaptation to concept drift. However, sustained operation of a stream learner

31 Aug 2026

A Shaky Voice Is Not Always a Dodge: Benchmarking Textual and Vocal Evasion Detection in Earnings Calls

Model ReleasesDGX agent

arXiv:2608.28040v1 Announce Type: new Abstract: Existing approaches to evasion detection in earnings calls focus on textual transcripts, treating evasion as a single-dimensional phenomenon. We argue t

28 Aug 2026

GLM-5.3-Flash Benchmarks on TensorSharp and llama.cpp

Model ReleasesDGX agent

Model: unsloth/GLM-5.3-Flash-GGUF UD-Q2_K_XL (4 shards, 101 GiB) + mmproj-BF16. Reference: llama.cpp PR #27754 (glm5next), CUDA 12.8, SM 120. Throughput Both engines back to back in one session, flash

MemToC: Benchmarking Memory-Tool Conflict Resolution in Large Language Models

Model ReleasesDGX agent

arXiv:2608.26295v1 Announce Type: cross Abstract: Tool-augmented LLMs must arbitrate between two fallible sources when a tool return conflicts with their parametric memory, yet existing evaluations me

Order Matters: A Chinese Multi-Panel Meme Benchmark for Vision-Language Reasoning

Model ReleasesDGX agent

arXiv:2608.26866v1 Announce Type: new Abstract: Many multimodal tasks depend on how visual elements are ordered and composed, not only on recognizing them in isolation. Internet memes are a compact ca

StreamAV-Bench: A Comprehensive Benchmark for Streaming Audio-Video Generation

Model ReleasesDGX agent

arXiv:2608.26336v1 Announce Type: cross Abstract: Recent advancements in generative models are pushing video generation toward unbounded streaming audio-video generation for real-time interactive worl

27 Aug 2026

CropCop: An Auditable 120-Class Plant-Health Model from Benchmark Reconstruction to a Quantised Runtime Artifact

Model ReleasesDGX agent

arXiv:2608.25539v1 Announce Type: new Abstract: A plant-health score can appear precise while resting on duplicated image families, a long-tailed label space, or a runtime file that was never evaluate

Towards Fine-Grained Text-to-3D Quality Assessment: A Benchmark and A Two-Stage Rank-Learning Metric

Model ReleasesDGX agent

arXiv:2509.23841v3 Announce Type: replace Abstract: Recent advances in Text-to-3D (T23D) generative models have enabled the synthesis of diverse, high-fidelity 3D assets from textual prompts. However,

26 Aug 2026

MatReplace: A Reference-Free, Conditioning-Aligned Benchmark for Material Replacement in Interior Scenes

Model ReleasesDGX agent

arXiv:2608.24107v1 Announce Type: cross Abstract: Material replacement is a common interior-design operation: changing the material of a selected surface while preserving its geometry, surroundings, a

Object Counting Across Modalities: Taxonomies, Benchmarks, Applications, and Open Challenges

Model ReleasesDGX agent

arXiv:2608.23845v1 Announce Type: new Abstract: Object-counting methods have rapidly shifted from class-specific density regression to open-vocabulary, foundation-model-backed counters. These methods

When Seeing Is Not Enough: Benchmarking Interactive Visual Grounding in LVLMs

ApplicationsDGX agent

arXiv:2608.23978v1 Announce Type: new Abstract: Visual grounding is typically evaluated as a one-shot mapping from an informative referring expression to a visual target. This formulation misses a cen

25 Aug 2026

AI Surrogate Modeling for Real-Time Tokamak Equilibrium Prediction: Benchmarking Neural Architectures and Validation on EXL-50U

Model ReleasesDGX agent

arXiv:2608.23217v1 Announce Type: cross Abstract: Fast and reliable plasma equilibrium prediction is essential for real-time tokamak operation and control, but conventional Grad-Shafranov (GS) solvers

CallScreenBench: Benchmarking Small Language Models as Phone Secretaries

Model ReleasesDGX agent

arXiv:2608.01033v2 Announce Type: replace-cross Abstract: Language models small enough to run on a handset, quantized to a few bits, are increasingly capable of acting on their user's behalf -- which

Cultural Moment Benchmark: Evaluating Video Cultural Reasoning and Grounding in Southeast Asia

Model ReleasesDGX agent

arXiv:2608.23065v1 Announce Type: cross Abstract: Cultural understanding in video means more than recognizing what is visible; it requires grasping the symbolic and temporal significance of cultural c

InstructMove: A Text-Indispensable Benchmark for Instruction-Following Manipulation

Model ReleasesDGX agent

arXiv:2608.22990v1 Announce Type: new Abstract: Vision-language-action (VLA) models have made general-purpose robot manipulation increasingly plausible by conditioning robot actions on natural-languag

LetzCross: A Cross-Lingual Page-Level Benchmark for Multimodal Retrieval over Luxembourgish Documents

Model ReleasesDGX agent

arXiv:2608.21714v1 Announce Type: new Abstract: Recent page-image retrievers such as ColPali have improved retrieval over visually rich documents, yet little is known about how they behave in cross-li

Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models

Model ReleasesDGX agent

arXiv:2508.07173v3 Announce Type: replace Abstract: Omni-modal Large Language Models (OLLMs) that integrate visual, auditory, and textual processing face severe safety risks. They exhibit fragile defe

SchemaGUI: A Schema-Driven Benchmark for Controllable GUI Generation Evaluation

Model ReleasesDGX agent

arXiv:2608.22390v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated strong potential in graphical user interface (GUI) generation, but reliable evaluation remains challengin

Signal or Noise? A Benchmark Study of Agent Skills in Web Development

Model ReleasesDGX agent

arXiv:2608.23067v1 Announce Type: new Abstract: Agent Skills are reusable procedural modules that are increasingly injected into coding-agent sessions to encode framework conventions, anti-patterns, a

WADE: A Reasoning-Annotated Benchmark for Multi-Instance Floating-Waste Grounding with Compact Vision-Language Models

Model ReleasesDGX agent

arXiv:2608.22950v1 Announce Type: new Abstract: Floating waste in inland waterways threatens aquatic ecosystems and requires timely monitoring under cluttered, multi-object conditions. Existing aquati

WildHandBench: A Benchmark for Handwritten Text Understanding that Challenges MLLMs and Humans

Model ReleasesDGX agent

arXiv:2608.22959v1 Announce Type: cross Abstract: While the top model on OmniDocBench now reaches 96.34% overall on printed-document parsing, the ability of current models to handle challenging handwr

24 Aug 2026

Benchmarking noisy label detection methods

Model ReleasesDGX agent

arXiv:2510.16211v2 Announce Type: replace Abstract: Label noise is a common problem in real-world datasets, affecting both model training and validation. Clean data are essential for achieving strong

22 Aug 2026

I benchmark DFlash 2 (PR build) in llama.cpp on Qwen 3.8 27B against all speculative methods for 3 days. 2.26x on 100 real coding prompts, 4.68x with one n-gram drafter on top. Up to 8x on specific cases.

Model ReleasesDGX agent

Hey guys, Inco AI shipped DFlash 2 a few days ago with a drafter for Qwen 3.8 27B and a llama.cpp PR. I built the PR and ran it against plain decoding, MTP, the n-gram lookup drafters, and my July DFl

21 Aug 2026

MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use

TutorialsDGX agent

arXiv:2608.20202v1 Announce Type: new Abstract: Memory has become a key component of large language models, enabling them to retain information and learn from long-term interactions. However, existing

NepOOC-M: Bilingual Nepali-English Benchmark and Comparative Analysis of Multimodal Architectures for OOC Detection

Model ReleasesDGX agent

arXiv:2608.19212v1 Announce Type: new Abstract: Out-of-context (OOC) misinformation pairs authentic images with misleading captions to construct false narratives without image manipulation, making det

20 Aug 2026

FairGlucose: A CGM Fairness Benchmark Reveals Subgroup Disparities Hidden in Population-Level Validation

Model ReleasesDGX agent

arXiv:2608.18296v1 Announce Type: cross Abstract: As CGM-based AI tools approach clinical deployment, whether their accuracy is equitable across patient demographics remains insufficiently tested. To

New benchmark just dropped!

Model ReleasesDGX agent

The pelican on a bicycle is sooo outdated, so I came up with a new, improved version. Qwen3.8-27b medium (UD-Q4_K_XL) vs. Sol 5.6 high vs. Qwen3.6-35B (UD-Q6_K_XL) Prompt (only real with typo!): 'Crea

OmniHandwritingOCR: A Diagnostic Benchmark for Evaluating Multimodal LLMs in Handwritten OCR Scenarios

Model ReleasesDGX agent

arXiv:2608.18586v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) are increasingly used as OCR systems in document and knowledge-processing pipelines, but their ability to fai

19 Aug 2026

OV3D-Bench: A Diagnostic Benchmark for Open-Vocabulary Monocular 3D Detection

Model ReleasesDGX agent

arXiv:2608.17110v1 Announce Type: new Abstract: Open-vocabulary monocular 3D detectors report strong in-domain performance, but each evaluates under a different protocol, several rely on per-image cat

18 Aug 2026

A Large-Scale Chinese Knowledge Graph-Text Alignment Dataset for Benchmarking Knowledge-Grounded LLMs

Model ReleasesDGX agent

arXiv:2510.06039v2 Announce Type: replace-cross Abstract: Reliable evaluation of knowledge-grounded Large Language Models (LLMs) in Chinese requires resources that explicitly align Chinese-language te

Automating Learner Assessment: Benchmarking Machine Learning and Deep Learning Models for EEG-Based Familiarity Prediction

Model ReleasesDGX agent

arXiv:2608.16541v1 Announce Type: cross Abstract: Objective assessment of learning remains a fundamental challenge in education. Electroencephalography (EEG) provides a direct, non-invasive window int

Benchmarking Identity-Sensitive LLM Outputs for Surveillance and Security Robots

Model ReleasesDGX agent

arXiv:2608.16030v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to generate textual robot design specifications, interaction policies, and risk assessments during ea

CBX-Bench: A Human-Aligned MLLM Council for Benchmarking Concept Bottleneck Model Explanations

Model ReleasesDGX agent

arXiv:2608.15404v1 Announce Type: new Abstract: Concept Bottleneck Models (CBMs) are designed to make visual classification interpretable by expressing predictions through human-understandable concept

Comprehensive Benchmarking of Deep Learning Architectures for Lung Cancer Histopathology

Model ReleasesDGX agent

arXiv:2608.15915v1 Announce Type: cross Abstract: Lung cancer remains the leading cause of cancer-related mortality worldwide, while histopathological diagnosis is often affected by inter-observer var

← Previous
1…4950515253…546
Next →