AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “benchmark”

GridTimelineEvolution
32,708 results
15 Jul 2026

Operationalising Multi-Dimensional Evaluation for Conversational Agents: A Scalable, Governed Pipeline with Selective Re-evaluation and Model Benchmarking

SafetyDGX agent

arXiv:2607.12085v1 Announce Type: new Abstract: Evaluating retail conversational agents requires methods beyond lexical-overlap metrics to assess intent alignment, factuality, helpfulness, clarity, to

10 Jul 2026

HumanForge: A Human-Centric Deepfake Video Benchmark with Multi-Agent Forgery Rationales

Model ReleasesDGX agent

arXiv:2607.08705v1 Announce Type: new Abstract: Rapid advancements in video diffusion models and temporal editing tools have enabled the generation of highly realistic human-centric videos, posing unp

UniRef-UAV: A Multimodal Benchmark for Universal Referring in UAV Imagery

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Model ReleasesDGX agent

arXiv:2607.08267v1 Announce Type: new Abstract: Unmanned aerial vehicles (UAVs) increasingly rely on visual grounding capabilities to localize task-relevant targets from diverse instructions in comple

9 Jul 2026

Does AI Understand Imaging? A Systematic Benchmark of Agentic AI for Computational Imaging Tasks

Model ReleasesDGX agent

arXiv:2607.07189v1 Announce Type: new Abstract: Vision-language models (VLMs) and agentic AI have shown strong performance on semantic visual tasks, but it remains unclear whether they can handle the

Generalist Vision-Language Models for Fast Radio Burst detection: a zero-shot benchmark against a specialized detector

Model ReleasesDGX agent

arXiv:2607.07382v1 Announce Type: new Abstract: Fast Radio Bursts (FRBs) are millisecond-duration radio transients whose automated detection increasingly relies on highly specialized deep learning mod

HAJJv2-CrowdCount: Zero-Shot Benchmark for Dense Crowd Counting

Model ReleasesDGX agent

arXiv:2607.07322v1 Announce Type: cross Abstract: Automated crowd counting in Hajj video is difficult not because current models lack capacity, but because the footage violates the assumptions those m

8 Jul 2026

Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving

Model ReleasesDGX agent

arXiv:2607.05399v1 Announce Type: cross Abstract: Large language model serving is increasingly limited by KV-cache growth under long-context workloads, yet existing KV-cache compression techniques are

GraphAllocBench: A Flexible Benchmark for Preference-Conditioned Multi-Objective Policy Learning

Model ReleasesDGX agent

arXiv:2601.20753v4 Announce Type: replace Abstract: Preference-Conditioned Policy Learning (PCPL) in Multi-Objective Reinforcement Learning (MORL) approximates diverse Pareto-optimal solutions by cond

HoloCount: A Holistic Visual Counting Benchmark for MLLMs

Model ReleasesDGX agent

arXiv:2607.06420v1 Announce Type: new Abstract: Visual counting is a fundamental pillar of multimodal intelligence, requiring a seamless integration of fine-grained grounding and spatial reasoning. Wh

PolyWorkBench: Benchmarking Multilingual Long-Horizon LLM Agents

Model ReleasesDGX agent

arXiv:2607.06008v1 Announce Type: new Abstract: Large language model (LLM) agents have shown strong performance in long-horizon tasks that require planning, tool use, and interaction with external env

RuBench: A Repository-Level Agentic Coding Benchmark with Natively Authored Russian Task Specifications

Model ReleasesDGX agent

arXiv:2607.06411v1 Announce Type: cross Abstract: Developers increasingly delegate real maintenance work to product-grade coding agents, and many state tasks in their native language, in the style of

UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation

Model ReleasesDGX agent

arXiv:2607.06306v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated growing competence in web page generation. However, existing text-driven approaches rely on complex pro

7 Jul 2026

A Fair Benchmarking of Deep Relational Database Learning Models

Model ReleasesDGX agent

arXiv:2607.03659v1 Announce Type: cross Abstract: Relational databases (RDBs) are the primary data infrastructure in many enterprises, yet recent deep learning methods designed for RDBs have been eval

APeB: Benchmarking Personalization Ability of Large Language Model Agents

Model ReleasesDGX agent

arXiv:2607.03162v1 Announce Type: new Abstract: LLM-powered agents struggle with personalization when users issue raw, underspecified queries. In this setting, agents must infer latent intent, extract

CausalGame: Benchmarking Causal Thinking of LLM Agents in Games

Model ReleasesDGX agent

arXiv:2607.04293v1 Announce Type: cross Abstract: Building AI Scientist agents with Large Language Models (LLMs) has recently attracted growing attention. Since scientific discovery fundamentally reli

Deriving Benchmarking Datasets from Long-Form Recordings: Challenges and Opportunities

ApplicationsDGX agent

arXiv:2607.03201v1 Announce Type: cross Abstract: Long-form recordings (LFRs) of child-centered audio are ecologically valid sources for studying early language development, but three problems limit t

EgoInertia-MI: A Multimodal Egocentric Vision and IMU Benchmark for Motor Impairment Assessment

Model ReleasesDGX agent

arXiv:2607.03934v1 Announce Type: new Abstract: Motor impairments, including tremor, bradykinesia, gait abnormalities, and postural instability, are common across many neurological and movement-relate

OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers

Model ReleasesDGX agent

arXiv:2607.04033v1 Announce Type: cross Abstract: Optimizer selection for large-scale model training has become a system-level design decision constrained jointly by compute, memory, tuning budget, an

PPE-Bench: A Benchmark for Evaluating MLLM Unlearning under Private-Public Entanglement

Model ReleasesDGX agent

arXiv:2607.02897v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have shown strong capabilities, but they may memorize private information from web data, raising privacy conc

Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain

Model ReleasesDGX agent

arXiv:2510.17801v2 Announce Type: replace-cross Abstract: Building robots that can perceive, reason, and act in dynamic, unstructured environments remains a central challenge. Recent embodied systems

Towards Open-World Referring Expression Comprehension: A Benchmark with Training-free Multi-task Consistency Checker

Model ReleasesDGX agent

arXiv:2605.25706v2 Announce Type: replace Abstract: Referring expression comprehension (REC) aims to localize a target object within an image based on a given expression. Although recent advances in v

TrendFact: A Benchmark Towards Hotspot Perception in Automatic Fact-Checking

Model ReleasesDGX agent

arXiv:2410.15135v5 Announce Type: replace Abstract: With the surge of online misinformation, Large Language Models (LLMs) and Reasoning Large Language Models (RLMs) serving as Automatic Fact-Checking

URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment

Model ReleasesDGX agent

arXiv:2607.04688v1 Announce Type: cross Abstract: Synthesis planning aiming to find pathways of reactions for a target molecule is one of the most important and challenging tasks in drug discovery. Re

XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection

Model ReleasesDGX agent

arXiv:2607.03562v1 Announce Type: new Abstract: As deepfake detection models increasingly produce natural language explanations, their reasoning often remains weakly grounded in visual artifacts, limi

3 Jul 2026

Beyond Pixel Diffs: Benchmarking Image Change Captioning for Web UI Visual Regression Testing

Model ReleasesDGX agent

arXiv:2607.01728v1 Announce Type: cross Abstract: Visual regression testing (VRT) is a standard quality assurance step in modern software release pipelines. On every change, it re-renders user interfa

Human Capital, Not Model Benchmarks, Predicts Hybrid Intelligence in Forecasting

Model ReleasesDGX agent

arXiv:2607.02467v1 Announce Type: cross Abstract: Whether pairing people with AI helps or hurts is usually reported as a single average effect. Using a real-money prediction market (Polymarket) as an

2 Jul 2026

AV-SyncBench: Decoupled Benchmarking of Temporal and Semantic Audio-Visual Synchronization

Model ReleasesDGX agent

arXiv:2607.00726v1 Announce Type: new Abstract: Audio-visual feature extraction is a fundamental component of multimodal understanding and generation tasks. However, existing evaluation protocols for

EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards

Model ReleasesDGX agent

arXiv:2607.00218v1 Announce Type: cross Abstract: Vision-language models (VLMs) are now proposed as runtime safety guards for embodied agents in homes and factories. A deployable guard must catch genu

EmbodimentSemantic: A Spatial Scene-Graph Dataset and Benchmark for Vision-Language Models on Embodied Manipulation Trajectories

Model ReleasesDGX agent

arXiv:2607.00020v1 Announce Type: new Abstract: Spatial grounding remains a key limitation of vision-language-action (VLA) systems for robotic manipulation. While current models can recognize objects

LongVQUBench: Benchmarking Long-Term Video Quality Understanding of Vision-Language Models

Model ReleasesDGX agent

arXiv:2607.01086v1 Announce Type: cross Abstract: The evaluation of long-term video quality understanding remains an open challenge for large vision-language models (LVLMs). Existing video quality ben

MemSyco-Bench: Benchmarking Sycophancy in Agent Memory

Model ReleasesDGX agent

arXiv:2607.01071v1 Announce Type: cross Abstract: Memory has emerged as a cornerstone of modern LLM-based agents, supporting their evolution from single-turn assistants to long-term collaborators. How

MoHallBench: A Benchmark for Motion Hallucination in Video Large Language Models

Model ReleasesDGX agent

arXiv:2607.01117v1 Announce Type: new Abstract: Video Large Language Models (VideoLLMs) have shown strong progress in video understanding, yet they still suffer from hallucinations that are inconsiste

MSQA: A Natively Sourced Multilingual and Multicultural SimpleQA Benchmark

Model ReleasesDGX agent

arXiv:2607.00724v1 Announce Type: new Abstract: Multilingual fluency often invites a stronger assumption: a model that can speak a user's language must also understand the culture encoded by that lang

PHREEQC-MCQ-200: A Diagnostic Benchmark for Tool-Augmented Scientific Simulator Agents

Model ReleasesDGX agent

arXiv:2607.00436v1 Announce Type: new Abstract: Large language model agents are increasingly connected to scientific software, yet it remains unclear when tool access makes scientific computation more

1 Jul 2026

Arena-T2I Hard: Benchmarking and Improving Faithfulness with Dependency-Aware Checklist

Model ReleasesDGX agent

arXiv:2606.31711v1 Announce Type: new Abstract: Faithfulness -- how precisely a generated image aligns with its prompt -- is increasingly central to the real-world utility of text-to-image (T2I) model

BLUEX v2: Benchmarking LLMs on Open-Ended Questions from Brazilian University Entrance Exams

Model ReleasesDGX agent

arXiv:2606.22723v2 Announce Type: replace Abstract: Although Large Language Models (LLMs) excel in many tasks, their assessment in Portuguese has received less attention, particularly for open-ended,

Bridging Scientific Heritage: An Arabic--Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer

Model ReleasesDGX agent

arXiv:2606.30943v1 Announce Type: new Abstract: Russian and Arabic are among the major languages of scientific communication. Language barriers impede the exchange of research results between these co

ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents

Model ReleasesDGX agent

arXiv:2606.31174v1 Announce Type: new Abstract: Production large language-model (LLM) agents are increasingly deployed not as lone problem-solvers but as managers: a main model creates specialized sub

CooperScene: Multi-Modal Cooperative Autonomy Benchmark with C-V2X Communication Characterization

Model ReleasesDGX agent

arXiv:2606.31219v1 Announce Type: new Abstract: Cellular vehicle-to-everything (C-V2X) enables cooperative perception, prediction, and planning beyond the field of view of individual agents. However,

M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding

Model ReleasesDGX agent

arXiv:2601.08758v4 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) reasoning has proven effective in enhancing large language models by encouraging step-by-step intermediate reasoning, a

OopsieVerse: A Safety Benchmark with Damage-Aware Simulation for Robot Manipulation

Model ReleasesDGX agent

arXiv:2606.31993v1 Announce Type: new Abstract: While robotic manipulation capabilities have advanced rapidly, physical safety remains a major barrier to deploying household robots: task success is in

PPT-Eval: A Benchmark for Computer-Use Agents on PowerPoint Tasks

Model ReleasesDGX agent

arXiv:2606.31154v1 Announce Type: cross Abstract: Creating and editing slides is a rich, multimodal activity that is ubiquitous in professional and educational settings, making it an ideal testbed for

Qualified Educational Capacity Planning under Heterogeneous Student Support Needs: A Synthetic Benchmark and Decision-Support Framework

Model ReleasesDGX agent

arXiv:2606.30650v1 Announce Type: cross Abstract: Educational support services often face a qualified-capacity problem: staff time is scarce, qualifications decay, new support needs can appear before

30 Jun 2026

AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World

Model ReleasesDGX agent

arXiv:2606.29716v1 Announce Type: new Abstract: This paper addresses the problem of monocular metric depth estimation in aerial UAV imagery. Although recent data-driven methods have achieved remarkabl

Benchmark AUC Is Not Deployable Reliability: A Cross-Dataset Audit of Off-the-Shelf Features for Surveillance Video Anomaly Detection

Model ReleasesDGX agent

arXiv:2606.29506v1 Announce Type: new Abstract: Automated 'suspicious behavior' flagging is a headline promise of AI surveillance, and the field reports high frame-level ROC-AUC on standard video anom

Benchmarking LLM Agents on Meta-Analysis Articles from Nature Portfolio

AgentsDGX agent

arXiv:2606.17041v4 Announce Type: replace Abstract: Meta-analysis is a demanding form of evidence synthesis that combines literature retrieval, PI/ECO-guided study selection, and statistical aggregati

Can AI Draw Science? A Benchmark for Evaluating Scientific Figure Generation by Text-to-Image and Multimodal Models

Model ReleasesDGX agent

arXiv:2606.28406v1 Announce Type: cross Abstract: Text-to-image and multimodal generative models are increasingly used to produce scientific figures such as mechanism diagrams, experimental-design sch

CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models

Model ReleasesDGX agent

arXiv:2501.14940v4 Announce Type: replace-cross Abstract: Aligning large language models (LLMs) with human values is essential for their safe deployment and widespread adoption. Current LLM safety ben

How Far Can You Get Without a GPU? A Systematic Benchmark of Lightweight Hallucination Detection Across Question Answering, Dialogue, and Summarisation

Model ReleasesDGX agent

arXiv:2606.29809v1 Announce Type: cross Abstract: Hallucination detection has become a pressing requirement for trustworthy AI deployment at scale. The most accurate detection methods depend on GPU-in

KrishokChat: A Citation-Grounded Dataset and Benchmark for Bengali Agricultural Advisory

Model ReleasesDGX agent

arXiv:2606.29243v1 Announce Type: new Abstract: We present KrishokChat, the first citation-grounded Bengali agricultural instruction-tuning dataset for crop advisory in low-resource settings. We estab

Multi-Agent Routing as Set-Valued Prediction: A WildChat Benchmark and Cost-Aware Evaluation

Model ReleasesDGX agent

arXiv:2606.28925v1 Announce Type: cross Abstract: Tool and agent routing from natural-language prompts is naturally a set-valued prediction problem: a single query may require multiple agents, while o

Preference-ASR: A Preference-Aware Test Set for Benchmarking ASR in the Era of Speech LLMs

ResearchDGX agent

arXiv:2606.29534v1 Announce Type: new Abstract: Popular ASR test sets adopt inconsistent conventions for numbers, disfluencies, entities, and casing, while standard normalizers erase the format distin

REPAIR-Bench: A Benchmark for Robot Error Perception And Interaction Recovery

Model ReleasesDGX agent

arXiv:2606.29937v1 Announce Type: new Abstract: Understanding how users perceive and respond to robot failures is essential for building robust and trustworthy robot systems. Prior work, however, (i)

RoAd-RL: A Unified Library and Benchmark for Robust Adversarial Reinforcement Learning

Model ReleasesDGX agent

arXiv:2606.29867v1 Announce Type: cross Abstract: Deep Reinforcement Learning (DRL) has achieved significant success in robotics and autonomous systems, yet remains vulnerable to adversarial perturbat

SAKE: Software Architectural Knowledge Evaluation Benchmark for Large Language Models

Model ReleasesDGX agent

arXiv:2606.29520v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used as assistants across the software development lifecycle, yet their ability to reason about software

Semantic Correspondence: Unified Benchmarking and a Strong Baseline

ResearchDGX agent

arXiv:2505.18060v4 Announce Type: replace Abstract: Establishing semantic correspondence is a challenging task in computer vision, aiming to match keypoints with the same semantic information across d

SHOVIR: A Benchmark for Evaluating Vision Shortcut Learning in Radiology Report Generation

Model ReleasesDGX agent

arXiv:2606.30201v1 Announce Type: cross Abstract: Current evaluation protocols for Vision-Language Models (VLMs) in Radiology Report Generation (RRG) rely on report-level metrics that measure lexical

SWITCH: Benchmarking Modeling and Handling of Tangible Interfaces in Long-horizon Embodied Scenarios

Model ReleasesDGX agent

arXiv:2511.17649v4 Announce Type: replace-cross Abstract: Tangible control interfaces (TCIs), such as appliance panels, remotes, elevators, and embedded GUIs, are a fundamental component of everyday h

The Human Creativity Benchmark

Model ReleasesDGX agent

arXiv:2606.30561v1 Announce Type: new Abstract: Modern AI evaluation frameworks treat evaluator disagreement as noise to be resolved. In creative domains, professional disagreement reflects genuine di

VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-On

Model ReleasesDGX agent

arXiv:2603.11734v2 Announce Type: replace Abstract: As virtual try-on (VTON) continues to advance, a growing number of real-world scenarios have emerged, pushing beyond the ability of the existing spe

← Previous
1…5253545556…546
Next →