AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
20 Apr 2026

Beyond MCQ: An Open-Ended Arabic Cultural QA Benchmark with Dialect Variants

Model ReleasesDGX agent

arXiv:2510.24328v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly used to answer everyday questions, yet their performance on culturally grounded and dialectal co

COMPASS: Benchmarking Constrained Optimization in LLM Agents

Model ReleasesDGX agent

arXiv:2510.07043v2 Announce Type: replace Abstract: Human decision-making often involves constrained optimization. As LLM agents are deployed to assist with real-world tasks like travel planning, shop

ECG-Lens: Benchmarking ML & DL Models on PTB-XL Dataset

Model ReleasesDGX agent

arXiv:2604.15822v1 Announce Type: cross Abstract: Automated classification of electrocardiogram (ECG) signals is a useful tool for diagnosing and monitoring cardiovascular diseases. This study compare

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

From Zero to Detail: A Progressive Spectral Decoupling Paradigm for UHD Image Restoration with New Benchmark

Model ReleasesDGX agent

arXiv:2604.15654v1 Announce Type: new Abstract: Ultra-high-definition (UHD) image restoration poses unique challenges due to the high spatial resolution, diverse content, and fine-grained structures p

MemEvoBench: Benchmarking Memory MisEvolution in LLM Agents

Model ReleasesDGX agent

arXiv:2604.15774v1 Announce Type: new Abstract: Equipping Large Language Models (LLMs) with persistent memory enhances interaction continuity and personalization but introduces new safety risks. Speci

RoleConflictBench: A Benchmark of Role Conflict Scenarios for Evaluating LLMs' Contextual Sensitivity

Model ReleasesDGX agent

arXiv:2509.25897v2 Announce Type: replace-cross Abstract: People often encounter role conflicts -- social dilemmas where the expectations of multiple roles clash and cannot be simultaneously fulfilled

TwoHamsters: Benchmarking Multi-Concept Compositional Unsafety in Text-to-Image Models

Model ReleasesDGX agent

arXiv:2604.15967v1 Announce Type: cross Abstract: Despite the remarkable synthesis capabilities of text-to-image (T2I) models, safeguarding them against content violations remains a persistent challen

17 Apr 2026

AD4AD: Benchmarking Visual Anomaly Detection Models for Safer Autonomous Driving

Model ReleasesDGX agent

arXiv:2604.15291v1 Announce Type: new Abstract: The reliability of a machine vision system for autonomous driving depends heavily on its training data distribution. When a vehicle encounters significa

Benchmarking Linguistic Adaptation in Comparable-Sized LLMs: A Study of Llama-3.1-8B, Mistral-7B-v0.1, and Qwen3-8B on Romanized Nepali

Model ReleasesDGX agent

arXiv:2604.14171v1 Announce Type: new Abstract: Romanized Nepali, the Nepali language written in the Latin alphabet, is the dominant medium for informal digital communication in Nepal, yet it remains

Benchmarking Optimizers for MLPs in Tabular Deep Learning

Model ReleasesDGX agent

arXiv:2604.15297v1 Announce Type: new Abstract: MLP is a heavily used backbone in modern deep learning (DL) architectures for supervised learning on tabular data, and AdamW is the go-to optimizer used

CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors

Model ReleasesDGX agent

arXiv:2604.14773v1 Announce Type: new Abstract: While LLMs have demonstrated remarkable potential in Question Answering (QA), evaluating personalization remains a critical bottleneck. Existing paradig

16 Apr 2026

DeEscalWild: A Real-World Benchmark for Automated De-Escalation Training with SLMs

Model ReleasesDGX agent

arXiv:2604.13075v1 Announce Type: new Abstract: Effective de-escalation is critical for law enforcement safety and community trust, yet traditional training methods lack scalability and realism. While

HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark

Model ReleasesDGX agent

arXiv:2604.13954v1 Announce Type: new Abstract: Existing agent-safety evaluation has focused mainly on externally induced risks. Yet agents may still enter unsafe trajectories under benign conditions.

LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Models

Model ReleasesDGX agent

arXiv:2511.11334v3 Announce Type: replace Abstract: The rapid advancement of large language models (LLMs) has not been matched by their evaluation in low-resource languages, especially Southeast Asian

LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning

Model ReleasesDGX agent

arXiv:2604.14140v1 Announce Type: new Abstract: As language models are increasingly deployed for complex autonomous tasks, their ability to reason accurately over longer horizons becomes critical. An

MERRIN: A Benchmark for Multimodal Evidence Retrieval and Reasoning in Noisy Web Environments

Model ReleasesDGX agent

arXiv:2604.13418v1 Announce Type: new Abstract: Motivated by the underspecified, multi-hop nature of search queries and the multimodal, heterogeneous, and often conflicting nature of real-world web re

Synthetic Tabular Generators Fail to Preserve Behavioral Fraud Patterns: A Benchmark on Temporal, Velocity, and Multi-Account Signals

Model ReleasesDGX agent

arXiv:2604.13125v1 Announce Type: new Abstract: We introduce behavioral fidelity -- a third evaluation dimension for synthetic tabular data that measures whether generated data preserves the temporal,

15 Apr 2026

Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving

Model ReleasesDGX agent

arXiv:2510.00919v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) with foundation models has achieved strong performance across diverse tasks, but their capacity for exper

CodeSpecBench: Benchmarking LLMs for Executable Behavioral Specification Generation

Model ReleasesDGX agent

arXiv:2604.12268v1 Announce Type: cross Abstract: Large language models (LLMs) can generate code from natural language, but the extent to which they capture intended program behavior remains unclear.

CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systems

Model ReleasesDGX agent

arXiv:2604.12312v1 Announce Type: new Abstract: As Large Language Models (LLMs) are increasingly deployed as task-oriented agents in enterprise environments, ensuring their strict adherence to complex

DeepTest Tool Competition 2026: Benchmarking an LLM-Based Automotive Assistant

ResearchDGX agent

arXiv:2604.12615v1 Announce Type: new Abstract: This report summarizes the results of the first edition of the Large Language Model (LLM) Testing competition, held as part of the DeepTest workshop at

League of LLMs: A Benchmark-Free Paradigm for Mutual Evaluation of Large Language Models

Model ReleasesDGX agent

arXiv:2507.22359v4 Announce Type: replace Abstract: Although large language models (LLMs) have shown exceptional capabilities across a wide range of tasks, reliable evaluation remains a critical chall

14 Apr 2026

AffordSim: A Scalable Data Generator and Benchmark for Affordance-Aware Robotic Manipulation

Model ReleasesDGX agent

arXiv:2604.11674v1 Announce Type: cross Abstract: Simulation-based data generation has become a dominant paradigm for training robotic manipulation policies, yet existing platforms do not incorporate

BareBones: Benchmarking Zero-Shot Geometric Comprehension in VLMs

Model ReleasesDGX agent

arXiv:2604.10528v1 Announce Type: new Abstract: While Vision-Language Models (VLMs) demonstrate remarkable zero-shot recognition capabilities across a diverse spectrum of multimodal tasks, it yet rema

Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method

Model ReleasesDGX agent

arXiv:2604.11209v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable success across a wide range of applications especially when augmented by external knowledge thro

GameplayQA: A Benchmarking Framework for Decision-Dense POV-Synced Multi-Video Understanding of 3D Virtual Agents

AgentsDGX agent

arXiv:2603.24329v2 Announce Type: replace-cross Abstract: Multimodal LLMs are increasingly deployed as perceptual backbones for autonomous agents in 3D environments, from robotics to virtual worlds. T

GazeVaLM: A Multi-Observer Eye-Tracking Benchmark for Evaluating Clinical Realism in AI-Generated X-Rays

Model ReleasesDGX agent

arXiv:2604.11653v1 Announce Type: new Abstract: We introduce GazeVaLM, a public eye-tracking dataset for studying clinical perception during chest radiograph authenticity assessment. The dataset compr

LARY: A Latent Action Representation Yielding Benchmark for Generalizable Vision-to-Action Alignment

Model ReleasesDGX agent

arXiv:2604.11689v1 Announce Type: new Abstract: While the shortage of explicit action data limits Vision-Language-Action (VLA) models, human action videos offer a scalable yet unlabeled data source. A

LVSum: A Benchmark for Timestamp-Aware Long Video Summarization

Model ReleasesDGX agent

arXiv:2604.10024v1 Announce Type: cross Abstract: Long video summarization presents significant challenges for current multimodal large language models (MLLMs), particularly in maintaining temporal fi

M2-Verify: A Large-Scale Multidomain Benchmark for Checking Multimodal Claim Consistency

Model ReleasesDGX agent

arXiv:2604.01306v2 Announce Type: replace Abstract: Evaluating scientific arguments requires assessing the strict consistency between a claim and its underlying multimodal evidence. However, existing

MMR-AD: A Large-Scale Multimodal Dataset for Benchmarking General Anomaly Detection with Multimodal Large Language Models

Model ReleasesDGX agent

arXiv:2604.10971v1 Announce Type: cross Abstract: In the progress of industrial anomaly detection, general anomaly detection (GAD) is an emerging trend and also the ultimate goal. Unlike the conventio

NetAgentBench: A State-Centric Benchmark for Evaluating Agentic Network Configuration

Model ReleasesDGX agent

arXiv:2604.09678v1 Announce Type: cross Abstract: As agentic network management gains popularity, there is a critical need for evaluation frameworks that transcend static, one-shot testing. To address

PepBenchmark: A Standardized Benchmark for Peptide Machine Learning

Model ReleasesDGX agent

arXiv:2604.10531v1 Announce Type: cross Abstract: Peptide therapeutics are widely regarded as the 'third generation' of drugs, yet progress in peptide Machine Learning (ML) are hindered by the absence

Seeing Through the Tool: A Controlled Benchmark for Occlusion Robustness in Foundation Segmentation Models

Model ReleasesDGX agent

arXiv:2604.11711v1 Announce Type: new Abstract: Occlusion, where target structures are partially hidden by surgical instruments or overlapping tissues, remains a critical yet underexplored challenge f

SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors

Model ReleasesDGX agent

arXiv:2510.17516v4 Announce Type: replace-cross Abstract: Large language model (LLM) simulations of human behavior have the potential to revolutionize the social and behavioral sciences, if and only i

Spatial Competence Benchmark

Model ReleasesDGX agent

arXiv:2604.09594v1 Announce Type: new Abstract: Spatial competence is the quality of maintaining a consistent internal representation of an environment and using it to infer discrete structure and pla

Turing Test on Screen: A Benchmark for Mobile GUI Agent Humanization

Model ReleasesDGX agent

arXiv:2604.09574v1 Announce Type: new Abstract: The rise of autonomous GUI agents has triggered adversarial countermeasures from digital platforms, yet existing research prioritizes utility and robust

13 Apr 2026

BEDTime: A Unified Benchmark for Automatically Describing Time Series

Model ReleasesDGX agent

arXiv:2509.05215v3 Announce Type: replace Abstract: Recent works propose complex multi-modal models that handle both time series and language, ultimately claiming high performance on complex tasks lik

Benchmarking CNN- and Transformer-Based Models for Surgical Instrument Segmentation in Robotic-Assisted Surgery

Model ReleasesDGX agent

arXiv:2604.09151v1 Announce Type: new Abstract: Accurate segmentation of surgical instruments in robotic-assisted surgery is critical for enabling context-aware computer-assisted interventions, such a

Bharat Scene Text: A Novel Comprehensive Dataset and Benchmark for Indian Language Scene Text Understanding

Model ReleasesDGX agent

arXiv:2511.23071v2 Announce Type: replace-cross Abstract: Reading scene text, that is, text appearing in images, has numerous application areas, including assistive technology, search, and e-commerce.

PilotBench: A Benchmark for General Aviation Agents with Safety Constraints

Model ReleasesDGX agent

arXiv:2604.08987v1 Announce Type: new Abstract: As Large Language Models (LLMs) advance toward embodied AI agents operating in physical environments, a fundamental question emerges: can models trained

QuanBench+: A Unified Multi-Framework Benchmark for LLM-Based Quantum Code Generation

Model ReleasesDGX agent

arXiv:2604.08570v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used for code generation, yet quantum code generation is still evaluated mostly within single frameworks

12 Apr 2026

Benchmark Your Local LLMs in 3 Commands

Model ReleasesDGX agent

This r/ollama post describes a streamlined method for performance-testing locally-running large language models using the Ollama framework, achievable with just three terminal commands. It likely intr

10 Apr 2026

A Benchmark of Classical and Deep Learning Models for Agricultural Commodity Price Forecasting on A Novel Bangladeshi Market Price Dataset

Model ReleasesDGX agent

arXiv:2604.06227v1 Announce Type: new Abstract: Accurate short-term forecasting of agricultural commodity prices is critical for food security planning and smallholder income stabilisation in developi

ASBench: Image Anomalies Synthesis Benchmark for Anomaly Detection

Model ReleasesDGX agent

arXiv:2510.07927v2 Announce Type: replace Abstract: Anomaly detection plays a pivotal role in manufacturing quality control, yet its application is constrained by limited abnormal samples and high man

LLM Spirals of Delusion: A Benchmarking Audit Study of AI Chatbot Interfaces

Model ReleasesDGX agent

arXiv:2604.06188v1 Announce Type: cross Abstract: People increasingly hold sustained, open-ended conversations with large language models (LLMs). Public reports and early studies suggest that, in such

16 Oct 2026

When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark

Model ReleasesDGX agent

arXiv:2610.21555v1 Announce Type: cross Abstract: Muon's gains in data efficiency motivate evaluating its performance under explicit data, time, and tuning budgets. On a single A100 GPU, we compare a

15 Oct 2026

An Explainable System for Offensive Content Detection in Nastaliq Urdu: Corpus Curation, Transformer Benchmarking, and Word-Level Attribution

TutorialsDGX agent

arXiv:2610.15795v1 Announce Type: cross Abstract: Automated moderation systems are deployed at scale, yet they remain opaque to the users and moderators affected by their decisions, and their coverage

ASIWorld: Benchmarking Autonomous Agents in a Persistent Survival Sandbox

AgentsDGX agent

arXiv:2610.14807v1 Announce Type: new Abstract: Evaluating autonomous agents requires examining whether they can translate goals into coherent action across extended interactions. In persistent enviro

BenchPreS: A Benchmark for Context-Aware Personalized Preference Selectivity of Persistent-Memory LLMs

Model ReleasesDGX agent

arXiv:2603.16557v2 Announce Type: replace-cross Abstract: Large language models (LLMs) increasingly store user preferences in persistent memory to support personalization across interactions. However,

14 Oct 2026

Embodied Skill Bench: Benchmarking How Well Agents with Embodied Skills Perform on Physical Tasks

Model ReleasesDGX agent

arXiv:2610.16238v1 Announce Type: new Abstract: LLM agents increasingly control robots using VLA policies, perception models, and procedural knowledge. These modules are difficult to reuse because the

Representation Geometry and Inductive Biases in Quantum Classifiers across Classical Visual Benchmarks

SafetyDGX agent

arXiv:2610.17722v1 Announce Type: cross Abstract: Quantum Machine Learning (QML) has a theoretical advantage over classical algorithms for specific synthetic data that is computationally hard. However

Skin Tone in the Wild: A Dataset and Benchmark for 10-tone MST Classification

Model ReleasesDGX agent

arXiv:2603.02475v2 Announce Type: replace Abstract: Skin tone classification is a fundamental task in computer vision, with applications in quantifying medical features, assessing demographic represen

What Matters in Quality-Aware VLA Pretraining: A Benchmark and Empirical Study

Model ReleasesDGX agent

arXiv:2610.18869v1 Announce Type: cross Abstract: How should demonstration quality be used during vision-language-action (VLA) pretraining to improve performance after target-task fine-tuning? We comp

12 Oct 2026

Benchmarking Embedding Geometries and Clustering Algorithms for Unsupervised Patent Claim Classification

ResearchDGX agent

arXiv:2610.12997v1 Announce Type: new Abstract: Prior-art retrieval systems are structurally organized around classification codes, making cross-code discovery difficult. This paper asks if an unsuper

JitterVLA: A Replay Benchmark for Motion Quality in VLA-Driven Human-Robot Manipulation

Model ReleasesDGX agent

arXiv:2610.13628v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies issue sparse targets whose arrival at the robot controller is delayed and irregular, so a reasonable predicted a

9 Oct 2026

FA-Bench: A Benchmark for Phone- and Word-Level Timestamp Accuracy in Forced Alignment and ASR on Clean and Noisy Speech

Model ReleasesDGX agent

arXiv:2609.32396v2 Announce Type: replace Abstract: Forced alignment estimates the timestamps of each word, phone or character in speech given its transcript. Published comparisons normalize transcrip

How Hackable Is Your Speech Quality Metric? A Corrected Protocol, a Benchmark, and What Patching Buys

Model ReleasesDGX agent

arXiv:2610.10899v1 Announce Type: new Abstract: Speech quality predictors are increasingly used as rewards, yet no agreed measure of their hackability exists. The usual measurement has two flaws. Firs

Large-Scale Benchmarking of Quantum Neural Network Configurations for Financial Time Series Forecasting

Model ReleasesDGX agent

arXiv:2610.12148v1 Announce Type: new Abstract: Quantum machine learning, and quantum neural networks (QNNs) in particular, are advancing fields with growing potential. Although systematic comparisons

The Review Lottery: Benchmarking an Observational Estimator of Peer-Review Noise (ICLR 2017-2025)

SafetyDGX agent

arXiv:2610.06591v2 Announce Type: replace Abstract: How much of a conference accept/reject decision would change if the same paper were reviewed by a different set of reviewers? Running a second indep

← Previous
1…7374757677…546
Next →