AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
14 Apr 2026

MMRareBench: A Rare-Disease Multimodal and Multi-Image Medical Benchmark

Model ReleasesDGX agent

arXiv:2604.10755v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have advanced clinical tasks for common conditions, but their performance on rare diseases remains largely unte

PaperScope: A Multi-Modal Multi-Document Benchmark for Agentic Deep Research Across Massive Scientific Papers

Model ReleasesDGX agent

arXiv:2604.11307v1 Announce Type: new Abstract: Leveraging Multi-modal Large Language Models (MLLMs) to accelerate frontier scientific research is promising, yet how to rigorously evaluate such system

VGA-Bench: A Unified Benchmark and Multi-Model Framework for Video Aesthetics and Generation Quality Evaluation

Model Releases
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
DGX agent

arXiv:2604.10127v1 Announce Type: cross Abstract: The rapid advancement of AIGC-based video generation has underscored the critical need for comprehensive evaluation frameworks that go beyond traditio

We benchmarked TranslateGemma against 5 other LLMs on subtitle translation across 6 languages. At first glance the numbers told a clean story, but then human QA added a chapter. [D]

ResearchDGX agent

This r/MachineLearning discussion post details a hands-on benchmark study in which TranslateGemma — Google's open translation model suite built on Gemma 3, available in 4B, 12B, and 27B sizes and cove

13 Apr 2026

A Benchmark of Dexterity for Anthropomorphic Robotic Hands

Model ReleasesDGX agent

arXiv:2604.09294v1 Announce Type: new Abstract: Dexterity is a central yet ambiguously defined concept in the design and evaluation of anthropomorphic robotic hands. In practice, the term is often use

MARINER: A 3E-Driven Benchmark for Fine-Grained Perception and Complex Reasoning in Open-Water Environments

Model ReleasesDGX agent

arXiv:2604.08615v1 Announce Type: cross Abstract: Fine-grained visual understanding and high-level reasoning in real-world open-water environments remain under-explored due to the lack of dedicated be

ParseBench is here!📊 We’ve just released ParseBench, an open benchmark + dataset for evaluating document parsing at scale. It includes: • 2…

Model ReleasesDGX agent

ParseBench is here!📊 We’ve just released ParseBench, an open benchmark + dataset for evaluating document parsing at scale. It includes: • 2,000+ human-reviewed enterprise documents • 167,000 evaluatio

10 Apr 2026

Continual Visual Anomaly Detection on the Edge: Benchmark and Efficient Solutions

Model ReleasesDGX agent

arXiv:2604.06435v1 Announce Type: cross Abstract: Visual Anomaly Detection (VAD) is a critical task for many applications including industrial inspection and healthcare. While VAD has been extensively

MM-MoralBench: A MultiModal Moral Evaluation Benchmark for Large Vision-Language Models

Model ReleasesDGX agent

arXiv:2412.20718v2 Announce Type: replace Abstract: The rapid integration of Large Vision-Language Models (LVLMs) into critical domains necessitates comprehensive moral evaluation to ensure their alig

Physical Knot Classification Beyond Accuracy: A Benchmark and Diagnostic Study

Model ReleasesDGX agent

arXiv:2603.23286v3 Announce Type: replace Abstract: Physical knot classification is a fine-grained task in which the intended cue is rope crossing structure, but high accuracy may still come from appe

This week @HumanXCo was a reminder that the AI conversation has moved on. No more benchmarks vs. bubble debate. The questions were practical…

ToolsDGX agent

This week @HumanXCo was a reminder that the AI conversation has moved on. No more benchmarks vs. bubble debate. The questions were practical: → How big do you need to be to train your own model? → How

9 Apr 2026

Anyhow, its not bad. Just not the vibe level that the benchmarks might indicate. And, for a first re-entry into the frontier model space, gi…

ApplicationsDGX agent

Anyhow, its not bad. Just not the vibe level that the benchmarks might indicate. And, for a first re-entry into the frontier model space, given the engineering efficiencies they achieved, it feels lik

16 Oct 2026

Benchmarking Efficient & Effective Camera Pose Estimation Strategies for Novel View Synthesis

Model ReleasesDGX agent

arXiv:2603.20428v2 Announce Type: replace Abstract: Novel view synthesis (NVS) approaches such as NeRFs or 3DGS can produce photo-realistic 3D scene representation from a set of images with known extr

BlenderWorld Benchmark from Alibaba

Model ReleasesDGX agent

Alibaba's Qwen team released a new repository on Wednesday, October 14, 2026. Its goal is to measure how well visual agents can create 3D scenes in Blender based on real-world references. https://gith

ChaosBench-Logic: A Benchmark for Logical and Symbolic Reasoning on Chaotic Dynamical Systems

Model ReleasesDGX agent

arXiv:2601.01982v2 Announce Type: replace Abstract: Large language models (LLMs) excel at natural language tasks but remain brittle in domains requiring precise logical and symbolic reasoning. Chaotic

Evo-DexBench: Benchmarking Generalization across Related Tasks for Dexterous Manipulation

Model ReleasesDGX agent

arXiv:2610.21180v1 Announce Type: new Abstract: General-purpose dexterous manipulation requires policies to transfer knowledge from related tasks to new tasks with only a few target demonstrations. Ex

From Adults to Children: Benchmarking Foundation Models for Chest X-Ray Classification under Domain Shift

Model ReleasesDGX agent

arXiv:2610.21809v1 Announce Type: new Abstract: Foundation models have shown significant potential across diverse fields and have been applied to medical imaging, but their robustness across patient p

RDL-Bench: A Benchmark for Task-Aware Question Answering over Relational Databases

Model ReleasesDGX agent

arXiv:2610.21382v1 Announce Type: new Abstract: Relational databases (RDBs) remain the primary infrastructure for storing and managing structured data across enterprise and scientific domains. Natural

Watts If? A Benchmark for Counterfactual Reasoning About the AI Data Center Buildout

Model ReleasesDGX agent

arXiv:2610.21397v1 Announce Type: new Abstract: An AI data center is committed years before it draws a watt: a few discrete choices (grid or on-site gas, evaporative or dry cooling, which county to fi

15 Oct 2026

Benchmarking 3D Reconstruction for Under-Ice Robotic Perception in Arctic Environments

Model ReleasesDGX agent

arXiv:2610.19097v1 Announce Type: new Abstract: As the Arctic transitions toward a seasonally ice-free state, capturing the complex morphology of submerged sea ice is crucial for advancing models of o

Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agents

Model ReleasesDGX agent

arXiv:2610.18052v1 Announce Type: new Abstract: Long-term memory has become a core component of large language model (LLM) agents. As agents accumulate personal information over long-term use, users i

Cross-Lingual TTS for Indian Languages: A Benchmark and Inference-Time Duration Estimation Strategies

Model ReleasesDGX agent

arXiv:2610.14826v1 Announce Type: cross Abstract: Recent alignment-free Text-to-Speech (TTS) models achieve high synthesis quality by removing explicit alignment modules and simplifying training. Howe

HM3D-M3: A Benchmark for Multi-Agent, Multi-Modal, Multi-Object Goal Navigation

Model ReleasesDGX agent

arXiv:2610.20121v1 Announce Type: new Abstract: Rapid progress in AI agents and robotics makes it a matter of the near future that humans are served by a team of robots. Such a team will receive sever

IMU-Bench: Benchmarking Wrist IMU Representations Across Kinematic, Semantic and Clinical Tasks

Model ReleasesDGX agent

arXiv:2610.19481v1 Announce Type: new Abstract: Wrist-worn inertial measurement units (IMUs) provide a common sensing modality for tasks ranging from activity and gesture recognition to pose estimatio

LED-BENCH: A Benchmark for Large-format Engineering Drawing Understanding

Model ReleasesDGX agent

arXiv:2610.19571v1 Announce Type: new Abstract: Real-world engineering drawings are often large-format and information-dense. Answering practical engineering questions therefore requires both fine-gra

MagicGarment: Benchmarking Single-Image Standalone 3D Garment Reconstruction

Model ReleasesDGX agent

arXiv:2610.19623v1 Announce Type: new Abstract: Single-image 3D reconstruction has advanced rapidly, yet standalone garments remain challenging due to thin-shell geometry, open boundaries, and layered

OLiVES: An Outdoor Low-Light Video Benchmark for Enhancement and Segmentation

Model ReleasesDGX agent

arXiv:2610.20501v1 Announce Type: new Abstract: Low-light video processing remains an underexplored problem in computer vision, primarily due to the scarcity of high-quality ground-truth data. Existin

SciRerankBench: Benchmarking Rerankers Towards Scientific Retrieval-Augmented Generation with LLMs

Model ReleasesDGX agent

arXiv:2508.08742v3 Announce Type: replace-cross Abstract: Scientific literature question answering is a pivotal step towards new scientific discoveries. Recently, extit{two-stage} retrieval-augmented

14 Oct 2026

Benchmarking Interaction-Oriented Aerial Mobility across Heterogeneous Environments

Model ReleasesDGX agent

arXiv:2610.18381v1 Announce Type: new Abstract: Aerial mobile manipulation requires unmanned aerial vehicles (UAVs) to traverse heterogeneous environments such as indoor and outdoor domains, and conne

BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views

Model ReleasesDGX agent

arXiv:2610.17457v1 Announce Type: cross Abstract: Browser agents need to make decisions and execute actions on behalf of users while consuming untrusted webpages, creating opportunities for malicious

DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments

Model ReleasesDGX agent

arXiv:2607.17291v2 Announce Type: replace Abstract: Deep research agents increasingly operate over the open web, where relevant records coexist with redundant summaries, outdated reports, and misleadi

I2O2I-Nav: A Benchmark with a Modular Agent for Long-Horizon Indoor-to-Outdoor-to-Indoor Navigation

Model ReleasesDGX agent

arXiv:2610.18307v1 Announce Type: new Abstract: Service and delivery robots often need to complete long-horizon navigation missions spanning indoor and outdoor environments. Existing embodied navigati

OmniRaindrop: A Comprehensive Benchmark for Dual-Focused Raindrop Removal under Day and Night Scenes

Model ReleasesDGX agent

arXiv:2510.17719v3 Announce Type: replace Abstract: Adherent raindrops on camera lenses are a pervasive source of real-world image degradation, undermining outdoor visual perception for downstream app

Sanzaru: Benchmarking Protocol Formation under Epistemic & Communication Asymmetry

Model ReleasesDGX agent

arXiv:2610.15163v1 Announce Type: cross Abstract: A language model's ability to solve a task individually does not ensure that a team can solve it under unequal access to information and communication

SidewalkBench: Benchmarking Visual Navigation on Urban Sidewalks

Model ReleasesDGX agent

arXiv:2606.16953v2 Announce Type: replace Abstract: Urban sidewalk navigation presents significant challenges due to complex structural layouts, dynamic pedestrian behaviors, and long distances. While

12 Oct 2026

From Benchmark to Bench: Can Agents Survive Real-World Drug Discovery?

Model ReleasesDGX agent

arXiv:2610.06411v2 Announce Type: replace Abstract: Agentic systems increasingly coordinate molecular-design tools, but it is unclear which layer of the stack limits outcomes on real projects. We deve

PPG-Bench: A Unified Benchmark for PPG Foundation Models

Model ReleasesDGX agent

arXiv:2610.13435v1 Announce Type: new Abstract: Photoplethysmography (PPG) is increasingly recorded by wearable and clinical devices at enormous scale, and a growing number of PPG foundation models (F

Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models

Model ReleasesDGX agent

arXiv:2610.12696v1 Announce Type: new Abstract: Vision-language-action models (VLAs) offer a promising route toward general-purpose robotics by transferring semantic knowledge from large-scale vision-

9 Oct 2026

Deep Learning vs. Statistical Models for Multi-Horizon Price Forecasting of Second-Hand Electronics: A Systematic Benchmark

Model ReleasesDGX agent

arXiv:2610.10727v1 Announce Type: cross Abstract: Forecasting resale prices of used electronics is critical for subscription-based platforms where pricing errors translate directly into risk. Unlike s

False Claims, Credible Images: A Red-Teaming Benchmark for Commercial Image Generators

Model ReleasesDGX agent

arXiv:2610.11112v1 Announce Type: cross Abstract: Image-generation models can now produce text-rich, natural-looking visual artifacts that are hard to distinguish from real-world evidence, such as new

GameCommBench: A Unified Benchmark and Type-Aware Evaluation for AI-Generated Game Commentary

Model ReleasesDGX agent

arXiv:2610.11129v1 Announce Type: new Abstract: Game commentary is an open-ended generation task requiring multimodal perception, strategic reasoning, and contextual knowledge. Existing AI-Generated G

How Is Automated Research Evaluated? A Survey of Benchmarks and Evaluation Practices

ResearchDGX agent

arXiv:2610.11877v1 Announce Type: new Abstract: Automated research systems support literature synthesis, ideation, experiments, writing, and peer review, but their evaluation is dispersed across tasks

MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcare

AgentsDGX agent

arXiv:2605.11814v2 Announce Type: replace Abstract: The large-scale deployment of personalized healthcare agents demands memory mechanisms that are exceptionally precise, safe, and capable of long-ter

Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild

Model ReleasesDGX agent

arXiv:2610.11328v1 Announce Type: new Abstract: Advances in foundation models are driving efforts to introduce agents to assist people in the physical world. Such agents require agentic spatial intell

PathLang: A Language-Centered Benchmark for Vision-Language Models in Computational Pathology

Model ReleasesDGX agent

arXiv:2610.11329v1 Announce Type: new Abstract: Pathology vision-language models (VLMs) have shown strong visual perception ability, but their robustness in the language domain remains poorly characte

RH-Detect: A Unified Benchmark for Reward Hacking Detection

Model ReleasesDGX agent

arXiv:2610.10947v1 Announce Type: new Abstract: Reward hacking, where a model exploits an evaluation signal without completing the intended task, threatens the reliability of deployed language model s

SAVU-BENCH: A Real-World Benchmark for Spatial Audio-Visual Understanding

Model ReleasesDGX agent

arXiv:2610.10624v1 Announce Type: cross Abstract: Spatial audio-visual understanding requires models to recognize not only what is present, but also where events occur and how they relate across modal

Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks

ResearchDGX agent

arXiv:2610.11978v1 Announce Type: new Abstract: Jev is a 'System One' model that returns a choice among given options instead of generating text. We study how such a specialized decision model compare

UXBench Pro: Benchmarking Personalized User Experience in Multi-Turn Dialogue Interactions

ResearchDGX agent

arXiv:2610.11638v1 Announce Type: new Abstract: Evaluating user experience (UX) with automated computational methods has gained increasing attention, supported by empirical evidence from UXBench. Howe

When Citations Mislead? A Claim-Level Benchmark for Legal Hallucination Detection

Model ReleasesDGX agent

arXiv:2610.10971v1 Announce Type: cross Abstract: Large language models are increasingly used in legal research and drafting, but they can still produce claims that sound convincing without being supp

8 Oct 2026

A Multi-Source Ultrasound Benchmark Revealing the Limits of Contemporary Self-Supervised Anomaly Detection Methods

Model ReleasesDGX agent

arXiv:2610.09677v1 Announce Type: new Abstract: Self-supervised anomaly detection is a promising paradigm for medical ultrasound, as normal images are often easier to obtain than exhaustive annotation

Arctic Questions, Missing Answers: A Dataset and Benchmark for LLM Abstention in Arctic Science

Model ReleasesDGX agent

arXiv:2610.09446v1 Announce Type: new Abstract: Large language models (LLMs) should abstain from scientific multiple-choice questions when no option is valid, but frequent abstention alone does not de

DISRQAD: Diffusion Image Super-Resolution Quality Assessment Dataset and Benchmark

Model ReleasesDGX agent

arXiv:2610.09077v1 Announce Type: new Abstract: Diffusion-based image super-resolution (SR) can create visually plausible detail that is not supported by the low-resolution input. We introduce DISRQAD

Hidden in Plain Sight: Benchmarking Agent Safety Against Decomposition Attacks with DECOMPBENCH

Model ReleasesDGX agent

arXiv:2606.13994v2 Announce Type: replace-cross Abstract: LLM-based Agents are becoming increasingly capable and widely deployed, creating growing incentives for adversarial misuse in the real-world.

Scaling Legal AI: Benchmarking Mamba and Transformers for Statutory Classification and Case Law Retrieval

Model ReleasesDGX agent

arXiv:2509.00141v2 Announce Type: replace-cross Abstract: Statutory corpora and judicial decisions are growing faster than legal professionals can read them, while individual judgments often exceed th

ScribbleEdit: A Benchmark for Scribble-Only Image Editing

Model ReleasesDGX agent

arXiv:2610.09382v1 Announce Type: cross Abstract: Scribble-based interaction provides a lightweight and intuitive way for users to specify image editing intents in interactive editing tools. However,

7 Oct 2026

DecepEval: A Benchmark for Evaluating Deception in LLM Agents

Model ReleasesDGX agent

arXiv:2610.07967v1 Announce Type: new Abstract: As large language model (LLM) agents become increasingly autonomous, they may pursue task performance through deception, raising concerns about their re

Decision model benchmark: Jev, Kev, Liquid d1, and more

Model ReleasesDGX agent

On 15 September, TypeSafe released Jev, and within a few weeks about 10 companies had shipped something that works the same way. OpenAI, Liquid, Cloudflare, AWS and PostHog all launched one, and an op

HARL-A: An Extensible Benchmark Framework for Heterogeneous Multi-Agent Adversarial Reinforcement Learning in IsaacLab

Model ReleasesDGX agent

arXiv:2510.01264v2 Announce Type: replace Abstract: Progress in adversarial multi-agent reinforcement learning (MARL) for robotics has been hampered by a lack of shared, extensible infrastructure that

The Failure Is in the Readout: Fine-Grained Emotion Recognition Benchmarks Measure Elicitation, Not Perception

Model ReleasesDGX agent

arXiv:2610.08162v1 Announce Type: cross Abstract: Fine-grained emotion recognition supports therapy tools and social robots, but it needs facial data, which raises privacy and data-protection concerns

← Previous
1…4647484950…546
Next →