AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
2 Jun 2026

Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends

AgentsDGX agent

arXiv:2606.01164v1 Announce Type: new Abstract: With rapid development of large language models and diffusion-based content generation, world modeling has attracted increasing research attention, bene

TravelEval: A Comprehensive Benchmarking Framework for Evaluating LLM-Powered Travel Planning Agents

Model ReleasesDGX agent

arXiv:2606.01046v1 Announce Type: new Abstract: The development of Large Language Models (LLMs) has significantly improved travel planning applications, yet evaluating such models is limited by existi

WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesis

Model ReleasesDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2606.01869v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly asked not only to write static interfaces, but to construct executable interactive worlds from natural lan

1 Jun 2026

A Visually Impaired Assistance Benchmark for VLM-as-a-Judge Evaluation

Model ReleasesDGX agent

arXiv:2605.31351v1 Announce Type: new Abstract: AI-based Visually Impaired Assistance (VIA) remains challenging, largely due to the high cost of human evaluation. The VLM-as-a-Judge paradigm may offer

AMNESIA: A Large Scale Medical Unlearning Benchmark Suite with Disease-Informed Analysis

Model ReleasesDGX agent

arXiv:2605.30599v1 Announce Type: cross Abstract: Medical knowledge is continuously evolving. This creates a need to update or selectively forget information encoded in already-trained medical LLMs. M

Benchmarking and Enhancing Text-to-Image Models for Generating Visual Representations in Early Arithmetic Education

Model ReleasesDGX agent

arXiv:2605.31212v1 Announce Type: cross Abstract: AI systems are increasingly used to support educational content creation, yet it remains unclear whether they can generate outputs that faithfully rep

BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs

Model ReleasesDGX agent

arXiv:2605.30900v1 Announce Type: new Abstract: Current multimodal models handle static image recognition well, but intuitive physical reasoning remains a weakness. Predicting how objects will move an

EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Vision

Model ReleasesDGX agent

arXiv:2605.31557v1 Announce Type: new Abstract: Continuous episodic memory is a core capability for autonomous agents operating in dynamic, real-world environments, yet current streaming video benchma

ERGeoBench:A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models

Model ReleasesDGX agent

arXiv:2605.31251v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have shown strong potential as embodied agents, yet embodied geo-localization remains underexplored due to th

HypoSpace: A Diagnostic Benchmark for Set-Valued Hypothesis Generation under Underdetermination and Sublinear Coverage Bounds

Model ReleasesDGX agent

arXiv:2510.15614v3 Announce Type: replace Abstract: Many scientific problems are underdetermined: multiple distinct hypotheses are equally consistent with the same observations. In such settings, effe

On the Robustness of Multilingual Text Embedding Rankings Across Learning Tasks, Languages, and Benchmark Datasets

Model ReleasesDGX agent

arXiv:2605.31142v1 Announce Type: cross Abstract: Large-scale multilingual text embedding models play crucial role in both research and industry, yet their behavior in language-specific, multi-task se

ReTabAD: A Benchmark for Restoring Semantic Context in Tabular Anomaly Detection

Model ReleasesDGX agent

arXiv:2510.02060v2 Announce Type: replace Abstract: In tabular anomaly detection (AD), textual semantics often carry critical signals, as the definition of an anomaly is closely tied to domain-specifi

SimulCost: A Cost-Aware Benchmark and Toolkit for Automating Physics Simulations with LLMs

Model ReleasesDGX agent

arXiv:2603.20253v2 Announce Type: replace-cross Abstract: Evaluating LLM agents for scientific tasks has focused on token costs while ignoring tool-use costs like simulation time and experimental reso

SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models

Model ReleasesDGX agent

arXiv:2605.31597v1 Announce Type: new Abstract: Measuring structured object understanding in vision foundation models remains challenging due to inconsistent evaluation protocols and limited part-leve

SPM-Bench: Benchmarking Large Language Models for Scanning Probe Microscopy

Model ReleasesDGX agent

arXiv:2602.22971v2 Announce Type: replace Abstract: As LLMs achieved breakthroughs in general reasoning, their proficiency in specialized scientific domains reveals pronounced gaps in existing benchma

TeachObs: A Human-Validated Benchmark for Multimodal Teaching Observation and Model Evaluation

Model ReleasesDGX agent

arXiv:2605.30673v1 Announce Type: new Abstract: Classroom videos contain observable teaching practices, but their pedagogical and visual signals are rarely organized in forms suitable for model evalua

29 May 2026

AttuneBench: A Conversation-Based Benchmark for LLM Emotional Intelligence

Model ReleasesDGX agent

arXiv:2605.21739v2 Announce Type: replace Abstract: Emotional intelligence (EI), the ability to perceive, understand, and respond appropriately to others' emotional states, is central to human communi

Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation

Model ReleasesDGX agent

arXiv:2605.28830v1 Announce Type: cross Abstract: As Large Language Models (LLMs) are increasingly deployed in safety-critical applications, robust content moderation becomes essential. We present a c

Beyond English and Evasion: A Human-Annotated Multi-Domain Benchmark for High-Stakes LLM Safety Evaluation in Chinese

Model ReleasesDGX agent

arXiv:2605.29667v1 Announce Type: new Abstract: When Large Language Models (LLMs) are deployed in Chinese-language settings, a troubling pattern emerges: safety systems that work well in English break

InsightEval: An Expert-Curated Benchmark for Assessing Insight Discovery in LLM-Driven Data Agents

Model ReleasesDGX agent

arXiv:2511.22884v2 Announce Type: replace Abstract: Data analysis has become an indispensable part of scientific research. To discover the latent knowledge and insights hidden within massive datasets,

MAGA-Bench: Machine-Augment-Generated Text via Alignment Detection Benchmark

Model ReleasesDGX agent

arXiv:2601.04633v2 Announce Type: replace Abstract: Machine-Generated Text (MGT) is becoming increasingly difficult to distinguish from Human-Written Text (HWT). This trend has exacerbated malicious a

MedCase-Structured: A Text-to-FHIR Dataset for Benchmarking Diagnostic Reasoning in Clinically Realistic EHR Settings

ResearchDGX agent

arXiv:2605.30295v1 Announce Type: cross Abstract: Large language models (LLMs) show promise for clinical reasoning and decision support, but evaluation in realistic, electronic health record-congruent

MusTBENCH: Benchmarking and Advancing Temporal Grounding in Music LLMs

Model ReleasesDGX agent

arXiv:2605.29300v1 Announce Type: cross Abstract: Recent Large Audio-Language Models (LALMs) have demonstrated promising abilities in understanding musical content. However, whether their responses ar

NICE: A Theory-Grounded Diagnostic Benchmark for Social Intelligence of LLMs

Model ReleasesDGX agent

arXiv:2605.29685v1 Announce Type: new Abstract: As large language models (LLMs) are increasingly applied in social contexts such as emotional companionship and customer service, measuring their social

OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfields

Model ReleasesDGX agent

arXiv:2605.29833v1 Announce Type: new Abstract: As multimodal language models play an increasingly important role in scientific research, materials science offers a critical testbed due to its interdi

Personalized Turn-Level User Conversation Satisfaction Benchmark

Model ReleasesDGX agent

arXiv:2605.29711v1 Announce Type: cross Abstract: User satisfaction with AI assistants is highly personalized: the same response may satisfy one user but disappoint another depending on what each user

28 May 2026

Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages

Model ReleasesDGX agent

arXiv:2510.05291v2 Announce Type: replace Abstract: As Large Language Models (LLMs) develop stronger multilingual capabilities, their sensitivity to culturally diverse entities becomes increasingly im

DynaSchedBench: Calibrated Dynamic Scheduling Benchmarks and Observability Paradox in LLM-based Scheduling Agents

Model ReleasesDGX agent

arXiv:2605.27566v1 Announce Type: new Abstract: Progress in neural combinatorial optimization for Dynamic Flexible Job Shop Scheduling Problem (DFJSP) is currently hindered by a methodological tension

Graph Neural Networks for Source Detection: A Review and Benchmark Study

Model ReleasesDGX agent

arXiv:2512.20657v2 Announce Type: replace-cross Abstract: The source detection problem arises when an epidemic process unfolds over a contact network, and the objective is to identify its point of ori

KSAFE-MM: A Multimodal Safety Benchmark via Localized Contextualization for Korean Cultural Risks

Model ReleasesDGX agent

arXiv:2605.28013v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) exacerbate safety risks by introducing vulnerabilities across multiple modalities, such as language and vision.

MeniOmni: A Structured Multimodal Benchmark for Holistic Meniscus Injury Assessment

Model ReleasesDGX agent

arXiv:2605.28161v1 Announce Type: new Abstract: Clinical diagnosis of meniscus injuries requires radiologists to integrate volumetric MRI evidence with patient context (e.g., sex, age, BMI) and to pro

MMTABREAL: Real-World Benchmark for Multimodal Table Understanding

Model ReleasesDGX agent

arXiv:2505.21771v2 Announce Type: replace-cross Abstract: Multimodal tables i.e. tabular layouts interleaved with charts, maps, icons, and color encodings are ubiquitous in real applications yet remai

Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Control

Model ReleasesDGX agent

arXiv:2601.15015v2 Announce Type: replace Abstract: Reinforcement learning (RL) has shown promising results in active flow control (AFC), yet progress in the field remains difficult to assess as exist

POINav: Benchmarking and Enhancing Final-Meters Arrival in Real-World Vision-Language Navigation

Model ReleasesDGX agent

arXiv:2605.28237v1 Announce Type: cross Abstract: Real-world navigation is fundamentally driven by Points of Interest (POIs), yet reaching a precise POI remains a critical 'final-meters' challenge. Ex

SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding

Model ReleasesDGX agent

arXiv:2601.21666v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) are a major focus of recent AI research. However, most prior work focuses on static image understanding, wh

27 May 2026

Benchmark Leakage Trap: Can We Trust LLM-based Recommendation?

Model ReleasesDGX agent

arXiv:2602.13626v3 Announce Type: replace Abstract: The expanding integration of Large Language Models (LLMs) into recommender systems poses critical challenges to evaluation reliability. This paper i

BESPOKE: Benchmark for Search-Augmented Large Language Model Personalization via Diagnostic Feedback

Model ReleasesDGX agent

arXiv:2509.21106v2 Announce Type: replace Abstract: Search-augmented large language models (LLMs) have advanced information-seeking tasks by integrating retrieval into generation, reducing users' cogn

Beyond Holistic Models: Systematic Component-level Benchmarking of Deep Multivariate Time-Series Forecasting

Model ReleasesDGX agent

arXiv:2605.26562v1 Announce Type: new Abstract: While previous research in multivariate time series forecasting has focused on developing complex holistic models, this work advocates for a shift towar

Drive-P2D: A Progressive Perception-to-Decision Benchmark for VLMs in Autonomous Driving

Model ReleasesDGX agent

arXiv:2601.14702v2 Announce Type: replace Abstract: Autonomous driving requires reliable perception and safe decision-making in complex scenarios. Recent vision-language models (VLMs) demonstrate reas

EconCausal: A Context-Aware Economic Reasoning Benchmark for Large Language Models

Model ReleasesDGX agent

arXiv:2510.07231v4 Announce Type: replace-cross Abstract: Socio-economic causal effects depend heavily on their institutional and environmental contexts. The same intervention can produce different, e

EpiQAL: Benchmarking Large Language Models in Epidemiological Question Answering and Reasoning

Model ReleasesDGX agent

arXiv:2601.03471v3 Announce Type: replace-cross Abstract: Reliable epidemiological reasoning requires synthesizing study evidence to infer disease burden, transmission dynamics, and intervention effec

FAB-Bench: A Framework for Adaptive RAG Benchmarking in Semiconductor Manufacturing

Model ReleasesDGX agent

arXiv:2605.26476v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) has become critical for knowledge-intensive applications, yet evaluating its performance in vertical domains remain

JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errors

Model ReleasesDGX agent

arXiv:2605.26955v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly deployed to users around the world, they are integrated into everyday tasks across diverse cultural c

Memory Architectures for Multi-Turn Text-to-SQL: A Benchmark and Empirical Study

Model ReleasesDGX agent

arXiv:2605.26394v1 Announce Type: new Abstract: Multi-turn Text-to-SQL is central to enterprise analytics yet remains predominantly evaluated in single-turn settings. We introduce EnterpriseMem-Bench,

ODOV: Benchmark the Open-Domain Open-Vocabulary Object Detection

Model ReleasesDGX agent

arXiv:2508.01253v2 Announce Type: replace Abstract: Existing studies typically investigate domain shift and category shift as independent problems, however, in real-world scenarios, the two types of s

ORLoopBench: Solver-in-the-Loop Benchmarks for Self-Correction and Behavioral Rationality in Operations Research

Model ReleasesDGX agent

arXiv:2601.21008v3 Announce Type: replace-cross Abstract: Operations Research practitioners debug infeasible models through an iterative process: inspecting Irreducible Infeasible Subsystems ( IIS), i

PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers

Model ReleasesDGX agent

arXiv:2605.26730v1 Announce Type: new Abstract: The rapid growth in submissions to machine learning venues has strained the scientific peer-review system and intensified interest in LLM-based automate

RoadGIE: Towards A Global-Scale Aerial Benchmark for Generalizable Interactive Road Extraction

Model ReleasesDGX agent

arXiv:2605.26862v1 Announce Type: new Abstract: Accurate road segmentation from aerial imagery is fundamental to many geospatial applications. However, existing datasets often suffer from limited scen

TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents

Model ReleasesDGX agent

arXiv:2601.05899v2 Announce Type: replace Abstract: Recent breakthroughs in Large Language Models (LLMs) have positioned them as a promising paradigm for agents, with long-term planning and decision-m

When Does Deep RL Beat Calibrated Baselines? A Benchmark Study on Adaptive Resource Control

Model ReleasesDGX agent

arXiv:2605.26418v1 Announce Type: cross Abstract: A properly calibrated rule-based autoscaler can beat every one of six mainstream deep reinforcement learning (DRL) algorithms on cost across every wor

26 May 2026

AstroMind: A High-Fidelity Benchmark for Spacecraft Behavior Reasoning Based on Large Language Models

Model ReleasesDGX agent

arXiv:2605.24573v1 Announce Type: new Abstract: Understanding why a spacecraft maneuvers -- rather than simply that it did -- is an increasingly important problem for space domain awareness as Earth o

Benchmarking Pathology Foundation Models for Spatial Domain Understanding

Model ReleasesDGX agent

arXiv:2605.25764v1 Announce Type: cross Abstract: Pathology foundation models (PFMs) have emerged as a core approach for learning transferable representations from whole slide images (WSIs), and they

DiscoverPhysics: Benchmarking LLMs for Out-of-the-Box Scientific Thinking

Model ReleasesDGX agent

arXiv:2605.26087v1 Announce Type: cross Abstract: Frontier LLMs now perform strongly across a wide range of physics evaluations, but it is hard to disentangle genuine reasoning from recall of establis

GlobalDentBench: A Multinational Benchmark for Evaluating LLM Clinical Reasoning in Dentistry with Expert Calibration

Model ReleasesDGX agent

arXiv:2605.24636v1 Announce Type: new Abstract: While large language models (LLMs) hold transformative potential for medicine, their reasoning robustness and safety in real-world clinical scenarios re

Identifying and Mitigating Systemic Measurement Bias in Production LLM Inference Benchmarks

SafetyDGX agent

arXiv:2605.24217v1 Announce Type: new Abstract: As Large Language Models (LLMs) transition from research environments to production deployments, evaluating their performance against strict Service Lev

Omissive Bias in Religious Representation: Benchmarking LLM Answers to Everyday Ethical Decision-making

Model ReleasesDGX agent

arXiv:2605.24319v1 Announce Type: new Abstract: As large language models become a default source of guidance on personal, moral, and existential questions, it matters whether they draw on the religiou

RAW: Robust Avatar Watermarking -- Benchmarking and Baseline

Model ReleasesDGX agent

arXiv:2605.23994v1 Announce Type: cross Abstract: Digital avatar watermarking presents unique challenges: avatars are routinely post-processed with background replacement, reframing, and format conver

Toward a Benchmark for Controllable Simulation of Imperfect Students with Large Language Models

Model ReleasesDGX agent

arXiv:2605.25601v1 Announce Type: cross Abstract: Teacher education requires deliberate practice with learners who exhibit identifiable strengths, weaknesses, and partial mastery. Large language model

When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents

Model ReleasesDGX agent

arXiv:2605.24069v1 Announce Type: cross Abstract: The rise of tool-using Large Language Model (LLM) agents, standardized by protocols like the Model Context Protocol (MCP), has unlocked unprecedented

25 May 2026

CaST-Bench: Benchmarking Causal Chain-Grounded Spatio-Temporal Reasoning for Video Question Answering

Model ReleasesDGX agent

arXiv:2605.23216v1 Announce Type: new Abstract: Cause-and-effect reasoning in video is a significant challenge for Vision-Language Models (VLMs), as it requires going beyond surface-level perception t

← Previous
1…5556575859…546
Next →