AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
24 Sept 2026

MultiVENT-Raw: A Benchmark for Retrieval and Reasoning over Raw Videos

Model ReleasesDGX agent

arXiv:2609.28437v1 Announce Type: new Abstract: Online information is increasingly consumed in video format. Much of this comes in the form of *raw video*: continuous footage taken on a cell phone, wi

SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems

Model ReleasesDGX agent

arXiv:2603.17373v2 Announce Type: replace Abstract: Large language models are rapidly being deployed as AI tutors, yet current evaluation paradigms assess problem-solving accuracy and generic safety i

WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents

Model ReleasesDGX agent

arXiv:2510.01354v2 Announce Type: replace-cross Abstract: Multiple prompt injection attacks have been proposed against web agents. At the same time, various methods have been developed to detect gener

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
23 Sept 2026

DUMA-Bench: A Dual-Control Multi-Agent Benchmark for Evaluating LLM Agent Security

Model ReleasesDGX agent

arXiv:2609.24662v1 Announce Type: new Abstract: LLM-based agents increasingly operate in environments where they interact with users, tools, and external systems. Yet most security evaluations assume

Label-Efficient Learning for Ground-Based Sky-Image Classification: A Benchmark of Transfer Learning, Active Learning, and Pseudo-Labeling on GCD

Model ReleasesDGX agent

arXiv:2609.26631v1 Announce Type: new Abstract: Accurate ground-based cloud classification is important for atmospheric monitoring, solar-energy forecasting, aviation weather assessment, and climate o

MatcherCompass: A Deployment-Aware Benchmark to Guide Image Matcher Selection in the Wild

Model ReleasesDGX agent

arXiv:2609.25688v1 Announce Type: new Abstract: Field robots operating across time of day and sensing modalities require accurate image correspondences within onboard time and resource budgets. Howeve

Passes Alone, Fails Together: Benchmarking Semantic Coordination in Parallel LLM-Agent Development

Model ReleasesDGX agent

arXiv:2609.25396v1 Announce Type: new Abstract: Parallel coding agents can produce patches that work alone but fail when merged. This happens when one agent changes an interface or rule that another a

Peerify: Benchmarking Peer-Review Claim Verification

Model ReleasesDGX agent

arXiv:2609.25046v1 Announce Type: new Abstract: Peer review plays a central role in scholarly publishing, yet verifying whether reviewer claims are supported by manuscript evidence remains a largely m

Trains but Doesn't Learn: A Post-Training Delivery Benchmark for LLM Agents as Forward-Deployed Engineers

Model ReleasesDGX agent

arXiv:2609.25237v1 Announce Type: cross Abstract: Post-training is becoming a service (PTaaS): a customer hands an operator data and a goal, and a forward-deployed engineer (FDE) returns a fine-tuned,

What is the Difference Between Me and You? Benchmarking the Quality Gap Between Human-Written and AI-Generated Code

Model ReleasesDGX agent

arXiv:2609.12708v2 Announce Type: replace-cross Abstract: AI coding assistants are becoming co-authors of production software, yet their evaluation centers on functional correctness, leaving open whet

WidgetVA: A Widget-Centric Framework and Benchmark for Agentic Visual Analytics

Model ReleasesDGX agent

arXiv:2609.24094v1 Announce Type: cross Abstract: Visual analytics (VA) enables sensemaking through interactive visualization, but effective analysis often requires experts to translate high-level int

22 Sept 2026

An Empirical Survey and Benchmark of Learned Distance Indexes for Road Networks

Model ReleasesDGX agent

arXiv:2602.04068v2 Announce Type: replace Abstract: The calculation of shortest-path distances in road networks is a core operation in navigation systems, location-based services, and spatial analytic

ARCGym: Benchmarking Deep Reinforcement Learning in Autonomous Robotic Colonoscopy

Model ReleasesDGX agent

arXiv:2609.22803v1 Announce Type: new Abstract: Simulations for learning-based autonomous colonoscopic navigation focus mainly on fully actuated capsule robots, failing to capture the contact-rich nav

Augmenting PID Control with Deep Reinforcement Learning: A Hybrid Approach to the Industrial Benchmark

Model ReleasesDGX agent

arXiv:2609.22584v1 Announce Type: new Abstract: As industrial processes grow in complexity, traditional Proportional-Integral-Derivative (PID) controllers are often insufficient for handling their non

Can Coding Agents Reproduce Official Statistics? Metadata, Retry Budget and the Limits of Execution Feedback in a Controlled Eurostat Benchmark

Model ReleasesDGX agent

arXiv:2609.22222v1 Announce Type: cross Abstract: Large language models can generate executable data-analysis code, but successful execution is not equivalent to a valid official-statistics result. Th

ECP-Bench: Benchmarking and Learning Entertainment Content Promotion with Foundation Models

Model ReleasesDGX agent

arXiv:2609.22150v1 Announce Type: cross Abstract: Content promotion spans a broad set of skills, from understanding content to forecasting its market reception. However, LLMs' ability to support such

ETH-TraceBench: A Large-Scale Event-Stream Benchmark for Ethereum DeFi under Temporal, Protocol, and Contract Shift

Model ReleasesDGX agent

arXiv:2609.23659v1 Announce Type: new Abstract: Ethereum decentralized finance (DeFi) provides a public, time-stamped record of transaction-level event streams, but the same public symbols can create

LegendBench: A Diagnostic Benchmark for Legend Understanding with Counterfactual Interventions

Model ReleasesDGX agent

arXiv:2609.24172v1 Announce Type: new Abstract: Legends are fundamental to chart understanding, as reliable interpretation requires correctly binding legend entries to corresponding visual marks. Whil

MCP-GRANITE Benchmark: GRANularity Interface TEsting for MCP-Based LLM Agents

Model ReleasesDGX agent

arXiv:2609.24161v1 Announce Type: cross Abstract: As LLM agents increasingly interact with external tools through standardized protocols such as MCP, tool-interface design becomes a critical yet under

MIS-Bench: Benchmarking Multimodal LLMs for Psychotherapeutic Interpersonal Skills Assessment

Model ReleasesDGX agent

arXiv:2609.22778v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly used as evaluators, yet their reliability in professional assessment tasks that require expert

ORDER: A Fictitious-World Benchmark for Domain-Adaptive Embodied AI

Model ReleasesDGX agent

arXiv:2609.22285v1 Announce Type: new Abstract: Adapting language models to new domains via continual pre-training raises a basic evaluation problem: if the training corpus overlaps with what the mode

The Effect of Quantization on Clinical Benchmarks: Accuracy and Safety Across Model Families

Model ReleasesDGX agent

arXiv:2609.22216v1 Announce Type: cross Abstract: Quantization enables deployment of large language models on resource-constrained clinical edge devices, but its effect on clinical accuracy and safety

Vox-Infinity: Benchmarking the Limits of Long-Context Spoken Language Models

Model ReleasesDGX agent

arXiv:2609.22452v1 Announce Type: new Abstract: Long-context understanding remains a fundamental challenge for large language models, as excessively long inputs often lead models to forget salient inf

21 Sept 2026

A benchmark dataset and baseline methods for four-dimensional STEM diffraction patterns

Model ReleasesDGX agent

arXiv:2609.21593v1 Announce Type: new Abstract: Four-dimensional scanning transmission electron microscopy (4D-STEM) records a two-dimensional diffraction pattern at each electron-probe position, yiel

Benchmarking World Models for Continual Learning on Compositional Tasks

Model ReleasesDGX agent

arXiv:2609.22055v1 Announce Type: new Abstract: A desirable property of a world model is the ability to learn continually across tasks, adapting to new environments without forgetting what the agent h

CIBuzzBench: A Benchmark for Cross-Lingual Understanding of Chinese Internet Buzzwords

Model ReleasesDGX agent

arXiv:2609.21722v1 Announce Type: cross Abstract: Chinese social media has generated a vast and continually evolving lexicon of internet buzzwords whose meanings are often non-literal and deeply roote

GameASG-Bench: Benchmarking Autonomous Software Generation for Game Development

Model ReleasesDGX agent

arXiv:2609.21293v1 Announce Type: new Abstract: Autonomous software generation (ASG) aims to turn human requirements into executable applications, but delivering these applications does not necessaril

mu^2-Bench: A Multilingual Machine Unlearning Benchmark

Model ReleasesDGX agent

arXiv:2609.20945v1 Announce Type: new Abstract: Undesired information such as harmful content and private data propagates through Multilingual Large Language Models (LLMs) via direct training and indi

Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction

Model ReleasesDGX agent

arXiv:2609.21392v1 Announce Type: new Abstract: Natural audio-visual interaction is emerging as an important interface for AI assistants, allowing users to communicate through speech and vision rather

OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation

Model ReleasesDGX agent

arXiv:2609.22069v1 Announce Type: new Abstract: Reference-to-video (R2V) generation is evolving toward increasingly general and versatile reference control, giving rise to the emerging paradigm of omn

OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialogue

Model ReleasesDGX agent

arXiv:2609.21465v1 Announce Type: cross Abstract: We define OmniVChat (Omni Video Chat) as the task of native audio-visual dialogue between a user and an omni model. In OmniVChat, omni models directly

PhysioBench: A Unified Benchmark for Physiological Signal Question Answering

Model ReleasesDGX agent

arXiv:2609.20836v1 Announce Type: new Abstract: Physiological signals support diverse clinical and monitoring tasks, yet existing physiological signal foundation models typically require task-specific

Semantic Calibration Prevails Where Token Confidence Fails: Benchmarking Long-Form Scientific QA

Model ReleasesDGX agent

arXiv:2602.00279v2 Announce Type: replace Abstract: Reliable uncertainty quantification (UQ) is essential for safe deployment of large language models (LLMs) in scientific question answering, where lo

18 Sept 2026

Benchmarking LLM Compliance with China AI Generated Content Regulations

Model ReleasesDGX agent

arXiv:2609.19989v1 Announce Type: new Abstract: The widespread adoption of LLMs has led to escalating content compliance risks. Prior works have contributed to addressing these risks in the English co

Benchmarking MLLMs via Cognitive Expected Scene Graph for Safety-Critical Visual Negation Understanding

Model ReleasesDGX agent

arXiv:2609.19767v1 Announce Type: new Abstract: True machine intelligence requires transcending passive pixel registration to master top-down functional reasoning over absent information via visual ne

From Intent to Action: Benchmarking LLM Safety in Vehicle Voice Command Authorization

Model ReleasesDGX agent

arXiv:2609.19630v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly integrated into vehicle voice assistants. But linking natural-language requests to vehicle functions creat

Multi-hour llama.cpp optimization experiments on Qwen MoE models, patches, benchmarks, and reproduction guides

Model ReleasesDGX agent

I had Astra run a multi-hour investigation into speeding up local MoE inference while keeping the model weights and quantization unchanged. The experiments covered Qwen3.6-35B-A3B and Qwen3.8 Flash-Ne

SAFARI: An Industrial Benchmark for LLM-Assisted Hazard Analysis and Risk Assessment

Model ReleasesDGX agent

arXiv:2609.20584v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly considered for safety-critical engineering, yet their reliability in regulated functional-safety workflows

SlugTrails: An Egocentric Benchmark for Floor Plan Localization in Large Buildings

Model ReleasesDGX agent

arXiv:2609.19876v1 Announce Type: new Abstract: Floor-plan-based indoor visual localization enables infrastructure-free positioning, but most methods are developed and evaluated in small residential e

17 Sept 2026

A3P5 NEMESIS Integrated Rover Design for Environmental Reconnaissance and Robotic Sampling with Reproducible Mobility Analysis and an External Data Machine Learning Calibration Benchmark

Model ReleasesDGX agent

arXiv:2609.18245v1 Announce Type: new Abstract: A3P5 NEMESIS is a four-wheel rover intended to combine remote inspection, environmental observation and lightweight manipulation within one serviceable

DynoFluxBench: Benchmarking Kinodynamic Space-Time Planners in Dynamic Environments

Model ReleasesDGX agent

arXiv:2609.18549v1 Announce Type: new Abstract: Robots that leave structured, static environments must plan motions that are kinodynamically feasible and safe among moving obstacles. However, there ar

ReFigBench: Benchmarking Scientific Figure Reconstruction as Editable PowerPoint Artifacts

Model ReleasesDGX agent

arXiv:2609.18844v1 Announce Type: new Abstract: Multimodal coding agents are expected to turn visual inputs into usable artifacts, and they act through a harness, the layer of tools, context managemen

TeleAntiFraud 2.0: A Refreshable, Profile-Grounded, and Audio-Based Benchmark for Telecom Fraud Detection

Model ReleasesDGX agent

arXiv:2609.18748v1 Announce Type: cross Abstract: Telecom fraud scripts evolve rapidly and are often designed to resemble routine service conversations, creating two key requirements for audio-based t

TTM-Bench: A Framework for Text-to-Music System Performance Benchmarking

SafetyDGX agent

arXiv:2609.18585v1 Announce Type: cross Abstract: Text-to-music (TTM) systems are increasingly used to generate musical audio from natural-language descriptions. Robust evaluation is therefore essenti

16 Sept 2026

Are We Grading Properly? Understanding Failure Modes in Medical Benchmarks

ResearchDGX agent

arXiv:2609.16023v1 Announce Type: new Abstract: Medical evaluation is shifting from static option-based questioning to realistic clinical scenarios with open-ended output modes. Grading these at scale

SciNLP: A Domain-Specific Benchmark for Full-Text Scientific Entity and Relation Extraction in NLP

Model ReleasesDGX agent

arXiv:2509.07801v5 Announce Type: replace Abstract: Structured information extraction from scientific literature is crucial for capturing core concepts and emerging trends in specialized fields. While

The Neverwhere Visual Parkour Benchmark Suite

Model ReleasesDGX agent

arXiv:2609.16443v1 Announce Type: cross Abstract: State-of-the-art visual locomotion controllers are increasingly capable at handling complex visual environments, making evaluating their real-world pe

15 Sept 2026

A derivative-fidelity failure mode in physics-informed neural networks: strengthened benchmark evidence from function-value training

Model ReleasesDGX agent

arXiv:2609.13171v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) use automatic differentiation to impose differential-equation residuals, but good agreement in function values

Accuracy Is Not Service: A Decision-Aware Benchmark for Intermittent-Demand Forecasting

Model ReleasesDGX agent

arXiv:2609.13840v1 Announce Type: new Abstract: A contract-logistics spare-parts operator is paid on order-level service: an order counts only if every requested line is fulfilled, yet forecasters are

Decentralized Multi-Robot Task Allocation Under Degraded Communication: A Benchmark of Performance, Reliability, and Computation

Model ReleasesDGX agent

arXiv:2609.13711v1 Announce Type: new Abstract: Selecting a decentralized Multi-Robot Task Allocation (MRTA) method for embedded deployment on autonomous platforms requires considering more than route

GroundBench: A Factorized, Counterfactual Benchmark for Locating VLM Affordance Failures

Model ReleasesDGX agent

arXiv:2609.13308v1 Announce Type: new Abstract: A companion evaluation found that naming the target part in a manipulation prompt increased action accuracy by 0.32-0.63 across eight vision-language mo

InterSocialBench: Benchmarking Human and LLM Preferences for Companion-Robot Social Behavior

Model ReleasesDGX agent

arXiv:2609.15455v1 Announce Type: new Abstract: Companion robots face everyday situations in which several feasible behaviors may be appropriate, yet different people prefer different responses. We in

Is my model perplexed for the right reason? Contrasting LLMs' Benchmark Behavior with Token-Level Perplexity

Model ReleasesDGX agent

arXiv:2603.29396v2 Announce Type: replace Abstract: Standard evaluations of Large language models (LLMs) focus on task performance, offering limited insight into whether correct behavior reflects appr

MoveBench: A Benchmark for Global-Scale Wildlife Movement Forecasting

Model ReleasesDGX agent

arXiv:2609.15780v1 Announce Type: new Abstract: Understanding and predicting wildlife movement is critical for ecology and conservation. While trajectory forecasting has advanced for human and vehicle

NoteVQA: Benchmarking VLMs on Real-Life Questions from Human Communities

AgentsDGX agent

arXiv:2609.15695v1 Announce Type: new Abstract: Vision-language models (VLMs) increasingly power consumer-facing AI search, yet evaluating them on the diversity of everyday visual questions remains ch

Time Series Analysis in Frequency Domain: A Survey of Open Challenges, Opportunities and Benchmarks

ResearchDGX agent

arXiv:2504.07099v5 Announce Type: replace-cross Abstract: Frequency-domain analysis has emerged as a powerful paradigm for time series analysis, offering unique advantages over traditional time-domain

Vulnerability Localization Benchmark: Measuring Agentic Security Analysis at Repository Scale

Model ReleasesDGX agent

arXiv:2609.15939v1 Announce Type: cross Abstract: Language-model agents increasingly operate over complete software repositories, yet cybersecurity evaluations primarily measure whether they can detec

What Makes a 3D Scene Editable? A Factorized Benchmark of Fidelity, Locality, Consistency, and Preservation

Model ReleasesDGX agent

arXiv:2609.14899v1 Announce Type: new Abstract: Neural 3D scene editing is often evaluated by semantic alignment alone, although a convincing result may alter unrelated content or become inconsistent

12 Sept 2026

Defining AI Agents: A Compendium of Criteria, Metrics, and Benchmarks

AgentsDGX agent

arXiv:2609.11018v1 Announce Type: new Abstract: The term agent in artificial intelligence lacks a standard definition, complicating the evaluation, comparison, and reproducibility of AI agent research

11 Sept 2026

SynCo: Synthetic Community-Aware Attributed Graph Generator for Graph Neural Network Benchmarking

Model ReleasesDGX agent

arXiv:2609.10742v1 Announce Type: cross Abstract: Graph Neural Networks (GNNs) are powerful models for handling attributed graphs in tasks such as classification, link prediction, and community detect

← Previous
1…6263646566…546
Next →