AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
25 May 2026

CRONOS: Benchmarking Counterfactual Physical Consistency in Video Models

Model ReleasesDGX agent

arXiv:2605.23699v1 Announce Type: new Abstract: Video prediction is increasingly viewed as a path toward generalizable world models, yet it remains unclear whether these systems learn underlying causa

DDX-TRACE: A Benchmark for Medical Diagnostic Trajectories in VLMs

Model ReleasesDGX agent

arXiv:2605.23629v1 Announce Type: new Abstract: Medical diagnosis is not a single prediction from a fully specified vignette. It is a sequential workup: clinicians decide what evidence to obtain, revi

MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks

Model ReleasesDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2601.14652v5 Announce Type: replace Abstract: While multi-agent systems (MAS) promise elevated intelligence through coordination of agents, current approaches to automatic MAS design under-deliv

PoisonForge: Task-Level Targeted Poisoning Benchmark for Instruction-Tuned LLMs

Model ReleasesDGX agent

arXiv:2605.23168v1 Announce Type: cross Abstract: When practitioners fine-tune LLMs on unvetted datasets, an adversary can exploit the data supply chain through task-level poisoning: inserting a small

ProtDBench: A Unified Benchmark of Protein Binder Design and Evaluation

Model ReleasesDGX agent

arXiv:2605.04118v2 Announce Type: replace-cross Abstract: Recent advances in de novo protein binder design have enabled increasing experimental validation, yet reported in silico metrics remain diffic

RoboSurg-VQA: A Multimodal Benchmark for Surgical Segmentation-Aware Visual Question Answering

Model ReleasesDGX agent

arXiv:2605.23068v1 Announce Type: new Abstract: Reliable visual understanding in robot-assisted and minimally invasive surgery (RMIS/MIS) demands more than accurate masks: in clinical practice, clinic

VideoOdyssey: A Benchmark for Ultra-Long-Context and Omni-Modal Video Understanding

Model ReleasesDGX agent

arXiv:2605.22907v1 Announce Type: new Abstract: Real-world long video understanding requires models to perform continuous tracking, information integration and memory retention over massive temporal s

23 May 2026

Cross-domain benchmarks reveal when coordinated AI agents improve scientific inference from partial evidence

Model ReleasesDGX agent

arXiv:2605.22300v1 Announce Type: cross Abstract: Scientific evidence often spans instruments, databases, and disciplines, so no single source records the full phenomenon. This makes it difficult to d

22 May 2026

Beyond Benchmark Islands: Toward Representative Trustworthiness Evaluation for Agentic AI

Model ReleasesDGX agent

arXiv:2603.14987v2 Announce Type: replace Abstract: Agentic AI systems increasingly act through tool-augmented, multi-step workflows whose failures (unsafe tool use, unauthorised actions, social harm)

Do Vision Models Encode Object-Level Semantic Relatedness? A Cognitive Psychology-Inspired Benchmark

Model ReleasesDGX agent

arXiv:1709.03806v2 Announce Type: replace Abstract: Modern vision models have achieved strong object-recognition performance, yet it remains unclear whether their representations encode object-level s

From Recognition to Reasoning: Benchmarking and Enhancing MLLMs on Real-World Receipt Document Understanding

Model ReleasesDGX agent

arXiv:2605.22413v1 Announce Type: new Abstract: Extracting structured information from visual documents (Visual Information Extraction, VIE) is a cornerstone of business automation. While recent Multi

GrandGuard: Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety

Model ReleasesDGX agent

arXiv:2605.20203v1 Announce Type: cross Abstract: As older adults increasingly use LLM-based chatbots for companionship and assistance, a safety gap is emerging. Older adults may face vulnerabilities

MM-Conv: A Multimodal Dataset and Benchmark for Context-Aware Grounding in 3D Dialogue

Model ReleasesDGX agent

arXiv:2605.21796v1 Announce Type: cross Abstract: Grounding language in the physical world requires AI systems to interpret references that emerge dynamically during conversation. While current vision

RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems

Model ReleasesDGX agent

arXiv:2510.13910v2 Announce Type: replace Abstract: Retrieval-Augmented Generation (RAG) mitigates key limitations of Large Language Models (LLMs)-such as factual errors, outdated knowledge, and hallu

RankJudge: A Multi-Turn LLM-as-a-Judge Synthetic Benchmark Generator

Model ReleasesDGX agent

arXiv:2605.21748v1 Announce Type: new Abstract: As interactive LLM-based applications are created and refined, model developers need to evaluate the quality of generated text along many possible axes.

RealUserSim: Bridging the Reality Gap in Agent Benchmarking via Grounded User Simulation

Model ReleasesDGX agent

arXiv:2605.20204v1 Announce Type: cross Abstract: LLM-based user simulation is the primary mechanism for end-to-end agent evaluation, yet simulated users are poor proxies for real humans: unconstraine

TransitLM: A Large-Scale Dataset and Benchmark for Map-Free Transit Route Generation

Model ReleasesDGX agent

arXiv:2605.22355v1 Announce Type: new Abstract: Public transit route planning traditionally depends on structured map infrastructure and complex routing engines, and no existing dataset supports train

VGenST-Bench: A Benchmark for Spatio-Temporal Reasoning via Active Video Synthesis

Model ReleasesDGX agent

arXiv:2605.22570v1 Announce Type: new Abstract: Spatio-temporal reasoning is a core capability for Multimodal Large Language Models (MLLMs) operating in the real world. As such, evaluating it precisel

21 May 2026

The Yes-Man Syndrome: Benchmarking Abstention in Embodied Robotic Agents

Model ReleasesDGX agent

arXiv:2605.20544v1 Announce Type: cross Abstract: Vision-language models (VLMs) are used as high-level planners for embodied agents, translating natural language instructions and visual observations i

VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence

Model ReleasesDGX agent

arXiv:2605.20676v1 Announce Type: new Abstract: Establishing a clear link between model predictions and the visual evidence that supports them is critical for transparency and reliability in multimoda

20 May 2026

Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth

Model ReleasesDGX agent

arXiv:2605.19797v1 Announce Type: new Abstract: Monocular depth estimation has improved significantly in recent years, driven by increasingly powerful models and large-scale training data. Predicted d

EgoBabyVLM: Benchmarking Cross-Modal Learning from Naturalistic Egocentric Video Data

Model ReleasesDGX agent

arXiv:2605.19130v1 Announce Type: cross Abstract: Children acquire language grounding with remarkable robustness from limited visuo-linguistic input in ways that surpass today's best large multimodal

FineBench: Benchmarking and Enhancing Vision-Language Models for Fine-grained Human Activity Understanding

Model ReleasesDGX agent

arXiv:2605.19846v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have demonstrated remarkable capabilities in general video understanding, yet they often struggle with the fine-grained

Quantifying the Generalization Gap in Seizure Detection: A Large-Scale Empirical Benchmark via the SzCORE Challenge

Model ReleasesDGX agent

arXiv:2505.18191v2 Announce Type: replace-cross Abstract: Reliable automatic seizure detection from long-term electroencephalography (EEG) remains an unsolved challenge, as current models often fail t

19 May 2026

A Pilot Benchmark for NL-to-FOL Translation in Planetary Exploration

Model ReleasesDGX agent

arXiv:2605.17911v1 Announce Type: new Abstract: Future planetary exploration envisions autonomous robotic agents operating under severe communication constraints, without global positioning, and with

BacktestBench: Benchmarking Large Language Models for Automated Quantitative Strategy Backtesting

Model ReleasesDGX agent

arXiv:2605.17937v1 Announce Type: cross Abstract: Quantitative backtesting is essential for evaluating trading strategies but remains hampered by high technical barriers and limited scalability. While

Benchmarking Recurrent Event-Based Object Detection for Industrial Multi-Class Recognition on MTevent

Model ReleasesDGX agent

arXiv:2603.21787v2 Announce Type: replace Abstract: Event cameras are attractive for industrial robotics because they provide high temporal resolution, high dynamic range, and reduced motion blur. How

ClawArena: Benchmarking AI Agents in Evolving Information Environments

Model ReleasesDGX agent

arXiv:2604.04202v2 Announce Type: replace-cross Abstract: AI agents deployed as persistent assistants must maintain correct beliefs as their information environment evolves. In practice, evidence is s

CT-DegradBench: A Physics-Informed Benchmark for CT Degradation Detection and Severity Estimation

Model ReleasesDGX agent

arXiv:2605.16431v1 Announce Type: new Abstract: Computed tomography (CT) images are frequently degraded by acquisition artifacts, including noise, blur, streaking, aliasing, and metal artifacts. Yet C

Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory

Model ReleasesDGX agent

arXiv:2511.20857v2 Announce Type: replace-cross Abstract: Statefulness is essential for large language model (LLM) agents to perform long-term planning and problem-solving. This makes memory a critica

From Pixels to Places: A Systematic Benchmark for Evaluating Image Geolocalization Ability in Large Language Models

Model ReleasesDGX agent

arXiv:2508.01608v2 Announce Type: replace Abstract: Image geolocalization, the task of identifying the geographic location depicted in an image, is important for applications in crisis response, digit

GenTS: A Comprehensive Benchmark Library for Generative Time Series Models

Model ReleasesDGX agent

arXiv:2605.17804v1 Announce Type: new Abstract: Generative models have demonstrated remarkable potential in time series analysis tasks, like synthesis, forecasting, imputation, etc. However, offering

How Good LLMs Are at Answering Bangla Medical Visual Questions? Dataset and Benchmarking

Model ReleasesDGX agent

arXiv:2605.18111v1 Announce Type: new Abstract: Recent advancements in Large Language Models (LLMs) and Large Vision Language Models (LVLMs) have enabled general-purpose systems to demonstrate promisi

MentalBench: A DSM-Grounded Benchmark for Evaluating Psychiatric Diagnostic Capability of Large Language Models

Model ReleasesDGX agent

arXiv:2602.12871v2 Announce Type: replace Abstract: Large language models (LLMs) have attracted growing interest as supportive tools for psychiatric assessment and clinical decision support. However,

MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likeness

Model ReleasesDGX agent

arXiv:2601.08118v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used as human simulators, both for evaluating conversational systems and for generating fine-tuning da

MLReplicate: Benchmarking Autonomous Research Systems for Machine Learning Reproducibility

Model ReleasesDGX agent

arXiv:2605.16616v1 Announce Type: new Abstract: Autonomous research systems capable of generating complete scientific manuscripts have advanced rapidly, yet robust and realistic evaluation frameworks

OmniPro: A Comprehensive Benchmark for Omni-Proactive Streaming Video Understanding

Model ReleasesDGX agent

arXiv:2605.18577v1 Announce Type: new Abstract: Omni-proactive streaming video understanding, i.e., autonomously deciding when to speak and what to say from continuous audio-visual streams, is an emer

On Improving Multimodal Pedestrian Trajectory Prediction with CVAE: A Study on Benchmark and Robot Data

Model ReleasesDGX agent

arXiv:2605.18262v1 Announce Type: new Abstract: Accurate pedestrian trajectory prediction is crucial for autonomous systems operating in complex environments, such as modular buses and delivery robots

OSWorld-Human: Benchmarking the Efficiency of Computer-Use Agents

Model ReleasesDGX agent

arXiv:2506.16042v2 Announce Type: replace Abstract: Generative AI is being leveraged to solve a variety of computer-use tasks involving desktop applications. State-of-the-art systems have focused sole

PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts

Model ReleasesDGX agent

arXiv:2605.17028v1 Announce Type: cross Abstract: Large language models (LLMs) hallucinate with confidence: their outputs can be fluent, authoritative, and simply wrong. In medical, legal, and scienti

REBAR: Reference Ethical Benchmark for Autonomy Readiness

Model ReleasesDGX agent

arXiv:2605.18423v1 Announce Type: new Abstract: As autonomous systems grow more advanced, objective metrics to evaluate their ethical and legal compliance are critical for informing end users of their

RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies

Model ReleasesDGX agent

arXiv:2603.04639v2 Announce Type: replace-cross Abstract: Memory is critical for long-horizon and history-dependent robotic manipulation. Such tasks often involve counting repeated actions or manipula

SomaliWeb v1: A Quality-Filtered Somali Web Corpus with a Matched Tokenizer and a Public Language-Identification Benchmark

Model ReleasesDGX agent

arXiv:2605.18232v1 Announce Type: cross Abstract: Somali is a Cushitic language of the Horn of Africa with ~25 million speakers, yet no documented dedicated Somali pretraining corpus with a companion

SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain

Model ReleasesDGX agent

arXiv:2605.17946v1 Announce Type: new Abstract: Multimodal large language models are increasingly used as agent backbones that understand multimodal inputs, plan retrieval actions, invoke external too

Temporal Decay of Co-Citation Predictability: A 20-Year Statute Retrieval Benchmark from 396M Ukrainian Court Citations

Model ReleasesDGX agent

arXiv:2605.17639v1 Announce Type: new Abstract: Co-citation structure is widely assumed to provide stable retrieval signal in legal information systems. We test this assumption longitudinally by const

UAVFF3D: A Geometry-Aware Benchmark for Feed-Forward UAV 3D Reconstruction

Model ReleasesDGX agent

arXiv:2605.17942v1 Announce Type: new Abstract: Feed-forward 3D reconstruction has recently demonstrated strong generalization across diverse scenes, yet its performance in UAV imagery remains underex

UniPPTBench: A Unified Benchmark for Presentation Generation Across Diverse Input Settings

Model ReleasesDGX agent

arXiv:2605.17356v1 Announce Type: new Abstract: Existing works typically focus on presentation generation under isolated input settings, whereas real-world use cases span diverse scenarios, including

Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoning

Model ReleasesDGX agent

arXiv:2601.06943v2 Announce Type: replace-cross Abstract: In real-world video question answering scenarios, videos often provide only localized visual cues, while verifiable answers are distributed ac

WinDeskGround: A Benchmark for Robust GUI Grounding in Complex Multi-Window Desktop Environments

Model ReleasesDGX agent

arXiv:2605.16402v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have revolutionized GUI automation, yet their efficacy is largely established on idealized, single-layer interf

18 May 2026

ChronoEarth-492K: A Large Scale and Long Horizon Spatiotemporal Hyperspectral Earth Observation Dataset and Benchmark

Model ReleasesDGX agent

arXiv:2605.15666v1 Announce Type: new Abstract: Hyperspectral imaging (HSI) provides dense spectral information for the Earth's surface, enabling material-level understanding of land cover and ecosyst

ELDOR: A Dataset and Benchmark for Illegal Gold Mining in the Amazon Rainforest

Model ReleasesDGX agent

arXiv:2605.15397v1 Announce Type: new Abstract: Illegal gold mining in the Amazon rainforest causes deforestation, water contamination, and long-term ecosystem disruption, yet remains difficult to mon

MorphoHELM: A Comprehensive Benchmark for Evaluating Representations for Microscopy-Based Morphology Assays

Model ReleasesDGX agent

arXiv:2605.15383v1 Announce Type: new Abstract: Microscopy images contain rich information about how cells respond to perturbations, making them essential to applications like drug screening. To quant

Rule2DRC: Benchmarking LLM Agents for DRC Script Synthesis with Execution-Guided Test Generation

Model ReleasesDGX agent

arXiv:2605.15669v1 Announce Type: new Abstract: Manufacturable chip layouts must satisfy thousands of geometry-based design rules, and design rule checking (DRC) enforces them by running executable DR

TSBOW -- Traffic Surveillance Benchmark for Occluded Vehicles Under Various Weather Conditions

Model ReleasesDGX agent

arXiv:2602.05414v2 Announce Type: replace Abstract: Global warming has intensified the frequency and severity of extreme weather events, which degrade CCTV signal and video quality while disrupting tr

15 May 2026

ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents

Model ReleasesDGX agent

arXiv:2605.14153v1 Announce Type: cross Abstract: Exploitation is not a binary event. It is a ladder of acquiring progressive capabilities, from executing a single buggy line of code to taking full co

Herculean: An Agentic Benchmark for Financial Intelligence

Model ReleasesDGX agent

arXiv:2605.14355v1 Announce Type: new Abstract: As AI agents improve, the central question is no longer whether they can solve isolated well-defined financial tasks, but whether they can reliably carr

JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation

Model ReleasesDGX agent

arXiv:2512.12772v2 Announce Type: replace-cross Abstract: Understanding videos inherently requires reasoning over both visual and auditory information. To properly evaluate Omni-Large Language Models

MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models

Model ReleasesDGX agent

arXiv:2605.14906v1 Announce Type: new Abstract: Memory is essential for large vision-language models (LVLMs) to handle long, multimodal interactions, with two method directions providing this capabili

MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring

Model ReleasesDGX agent

arXiv:2510.23477v2 Announce Type: replace Abstract: Effective math tutoring requires not only solving problems but also diagnosing students' difficulties and guiding them step by step. While multimoda

NeuroTrain: Surveying Local Learning Rules for Spiking Neural Networks with an Open Benchmarking Framework

ResearchDGX agent

arXiv:2605.15058v1 Announce Type: cross Abstract: The rapid expansion of spiking neural networks (SNNs) has led to a proliferation of training algorithms that differ widely in biological inspiration,

← Previous
1…5657585960…546
Next →