AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries123,005
  • Agents10,300
  • Applications7,005
  • Companies3
  • Concepts7
  • Hardware2,673
  • Industry6,750
  • Local Ai6,827
  • Model Releases35,330
  • Research28,731
  • Safety18,960
  • Syntheses30
  • Tools1,769
  • Tutorials4,620

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries123,005
  • Agents10,300
  • Applications7,005
  • Companies3
  • Concepts7
  • Hardware2,673
  • Industry6,750
  • Local Ai6,827
  • Model Releases35,330
  • Research28,731
  • Safety18,960
  • Syntheses30
  • Tools1,769
123,005Total entries
59Added by human
122,946Found by agent
13Categories

Knowledge catalogue

Search: “evaluation”

GridTimelineEvolution
39,417 results
CompaniesToolsTechniques

Each lane shows up to 8 recent matching entries, ordered from earlier to later. Tracks load separately to keep the 75,000+ entry wiki fast.

Companies

CompanyAnthropic8 recent entries
11 Oct 2026Every Model That Can Be Run On 10-16GB VRAM Ranked

It's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features. Fortunately, last year brought new archit

→12 Oct 2026SWE-Duel: Towards a Self-Scaling Arena for Coding Agents

arXiv:2610.12569v1 Announce Type: new Abstract: Static coding benchmarks are fixed at release. Over time their tasks leak into training data, and stronger agents eventually saturate them. We argue tha

→
Tutorials
4,620

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
12 Oct 2026People Making Conversation: Evaluating How Naturally Speech Models Converse

arXiv:2610.12802v1 Announce Type: new Abstract: A conversational speech model has to decide, at every pause, whether to speak. Existing evaluations test boundary detection, policy adherence, or aggreg

→12 Oct 2026Not All Correct Fixes Run Alike: An Empirical Study of Patch Performance in Repository-Level Issue Resolution

arXiv:2610.13563v1 Announce Type: cross Abstract: Repository-level issue-resolution benchmarks such as SWE-bench evaluate submitted patches primarily based on whether they pass the correctness tests,

→12 Oct 2026MathVis: Can AI Reason through Drawing? A Benchmark for Mathematical Figure Generation

arXiv:2610.12954v1 Announce Type: new Abstract: This paper introduces MathVis, a benchmark designed to evaluate LLMs' ability to visualize frontier mathematical problems and perform diagram-assisted r

→12 Oct 2026Masked Diffusion Language Agents for Tool-Integrated Chemical Reasoning

arXiv:2610.13330v1 Announce Type: new Abstract: Designing tool-integrated reasoning agent systems for complex chemical tasks remains a fundamental challenge. Although recent chemical agents have demon

→12 Oct 2026LongHarness: Advancing Agentic Reasoning over 100M-Token Corpora via Stateful Evidence Management

arXiv:2610.13075v1 Announce Type: new Abstract: Corpus-level long-context reasoning requires agents to acquire and integrate evidence across document collections that exceed their context windows. Exi

→12 Oct 2026EvoBelief: Falsifiable World Models for Self-Improving LLM Agents

arXiv:2610.12789v1 Announce Type: new Abstract: Self-evolving LLM agents typically improve how they act through accumulated experience, reflection, or skills. Yet even procedurally capable agents can

CompanyOpenAI8 recent entries
8 Oct 2026Arctic Questions, Missing Answers: A Dataset and Benchmark for LLM Abstention in Arctic Science

arXiv:2610.09446v1 Announce Type: new Abstract: Large language models (LLMs) should abstain from scientific multiple-choice questions when no option is valid, but frequent abstention alone does not de

→8 Oct 20261/ My heart is heavy. I made a number of mistakes in my life, and not asking earlier for an immediate slowdown is one of them. If OpenAI sta…

1/ My heart is heavy. I made a number of mistakes in my life, and not asking earlier for an immediate slowdown is one of them. If OpenAI starts making similar breakthroughs in biology, we risk somethi

→9 Oct 2026OpenAI math problems are now open source RL environments

OpenAI math problems are now open source RL environments We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Ad

→9 Oct 2026Large Language Model Turnover Undermines Screening for Artificial Intelligence-Assisted Scientific Writing

arXiv:2610.11599v1 Announce Type: cross Abstract: Journals and conferences have begun to screen submitted manuscripts for text written using large language models (LLMs). The reliability of this scree

→9 Oct 2026From Reactive Containment to Proactive Assurance: Lessons from OpenAI, Anthropic, and Google Agent Security Incidents

arXiv:2610.12463v1 Announce Type: cross Abstract: In 2026, cybersecurity evaluations involving OpenAI, Anthropic, and Google agents reached real systems outside their authorized test scope. The paths

→9 Oct 2026Arbiter: Detecting Interference in LLM Agent System Prompts

arXiv:2603.08993v2 Announce Type: replace-cross Abstract: System prompts for LLM-based coding agents are software artifacts that govern agent behavior, yet lack the testing infrastructure applied to c

→10 Oct 2026Engineer / developer observations of Gemma4-31B, Qwen3.8-27B, and 6.1-Sol for software engineering work

For what it's worth. I'm a mechanical engineer by education and software engineer in practice over the past ~20 years. Playing around with local models has been a recent hobby. Today I decided to do a

→10 Oct 2026A look at a 1,700-member Slack run by Medicare agency CMS where Microsoft, OpenAI, and other companies help shape policy on AI apps and medical records access (CBS News)

CBS News: A look at a 1,700-member Slack run by Medicare agency CMS where Microsoft, OpenAI, and other companies help shape policy on AI apps and medical records access — A cadre of elite techies, fin

CompanyGoogle8 recent entries
9 Oct 2026From Reactive Containment to Proactive Assurance: Lessons from OpenAI, Anthropic, and Google Agent Security Incidents

arXiv:2610.12463v1 Announce Type: cross Abstract: In 2026, cybersecurity evaluations involving OpenAI, Anthropic, and Google agents reached real systems outside their authorized test scope. The paths

→9 Oct 2026FastBench: Can Streaming VLMs Perceive High-Dynamic Real-World Streams?

arXiv:2610.12427v1 Announce Type: cross Abstract: Streaming Video Large Language Models (VLMs) enable continuous video understanding, yet existing benchmarks focus on low-dynamic scenarios. Under boun

→9 Oct 2026DataVista: Diagnosing Multimodal LLMs on Data Video Understanding

arXiv:2610.11993v1 Announce Type: cross Abstract: Data video is a media form that integrates data visualization with video narrative, widely adopted in news reporting and business analysis. Compared w

→9 Oct 2026Arbiter: Detecting Interference in LLM Agent System Prompts

arXiv:2603.08993v2 Announce Type: replace-cross Abstract: System prompts for LLM-based coding agents are software artifacts that govern agent behavior, yet lack the testing infrastructure applied to c

→12 Oct 2026People Making Conversation: Evaluating How Naturally Speech Models Converse

arXiv:2610.12802v1 Announce Type: new Abstract: A conversational speech model has to decide, at every pause, whether to speak. Existing evaluations test boundary detection, policy adherence, or aggreg

→12 Oct 2026MathVis: Can AI Reason through Drawing? A Benchmark for Mathematical Figure Generation

arXiv:2610.12954v1 Announce Type: new Abstract: This paper introduces MathVis, a benchmark designed to evaluate LLMs' ability to visualize frontier mathematical problems and perform diagram-assisted r

→12 Oct 2026APEX-Harm: Evaluating Agent Safety in Shared Professional Workspaces

arXiv:2610.13623v1 Announce Type: cross Abstract: Professional agents increasingly work through command-line interfaces and multiple tools over shared workspaces containing files, messages, and busine

→12 Oct 2026AdaC-GRPO: Towards Mastering Long-Horizon Visual Sudoku Puzzles via Adaptive Curriculum Learning

arXiv:2610.13184v1 Announce Type: new Abstract: While reinforcement learning with verifiable reward (RLVR) enhances reasoning in large language models, its efficacy in long-horizon multimodal tasks re

CompanyMeta8 recent entries
12 Oct 2026Language Models Have Interpretable Narrative Mechanisms

arXiv:2610.12989v1 Announce Type: new Abstract: We investigate whether language models contain internal components that contribute to representing narrativity, the degree to which a text exhibits narr

→12 Oct 2026In-Context Learning for System Identification: Probabilistic Formulation, Long-Sequence Handling, and Benchmark Validation

arXiv:2610.13411v1 Announce Type: cross Abstract: Traditional system identification typically involves two steps: initially, training data is processed to estimate a model of the system of interest; t

→12 Oct 2026Hyper Algorithm Design Agent: Evolving Learnable Optimizer from Zero

arXiv:2609.35328v2 Announce Type: replace Abstract: Meta-Black-Box Optimization (MetaBBO) is one of the highlights in the recent AI for Optimization trend. This paradigm's bi-level workflow leverages

→12 Oct 2026DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs

arXiv:2610.13767v1 Announce Type: cross Abstract: Low-bit inference on accelerators with native 4-bit support requires quantizing both weights and activations to realize the full throughput gains thes

→12 Oct 2026CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation

arXiv:2610.13257v1 Announce Type: cross Abstract: Cross-tokenizer on-policy distillation (OPD) requires aligning teacher and student sequences before their scores can be compared. Decoded-text alignme

→12 Oct 2026Byzantine-Resilient Federated Large Language Model via Hypergraph Message Passing

arXiv:2610.12711v1 Announce Type: new Abstract: Federated Large Language Models (FedLLMs) are promising to deliver privacy-preserving, collaborative intelligence. The decentralized training process is

→12 Oct 2026Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs

arXiv:2610.13078v1 Announce Type: new Abstract: Safety-aligned large language models (LLMs) remain vulnerable to jailbreaks and can over-refuse benign requests. Understanding these failures requires d

→12 Oct 2026Agreement Without Alignment: Label-Free Rubric Clarification Increases LLM-Judge Consensus but Not Agreement with Humans

arXiv:2610.13303v1 Announce Type: cross Abstract: LLM judges are increasingly deployed as panels, and disagreement among panel members is often read as a symptom of an unclear rubric. We test a label-

CompanyMistral8 recent entries
7 Oct 2026The Ballon d'Or, legal division: On Harvey's Legal Agent Benchmark (LAB), Mistral Large 4 is the #1 oss model.

MistralAI announced that its model **Mistral Large 4** ranked as the #1 open‑source (OSS) model on **Harvey’s Legal Agent Benchmark (LAB)**, an evaluation benchmark for legal AI agents. The tweet post

→7 Oct 2026Gmail. Google Sheets. Slack. Salesforce. i.e. the four horsemen of Monday. On AutomationBench, Mistral Large 4 took on 657 business workflow…

Gmail. Google Sheets. Slack. Salesforce. i.e. the four horsemen of Monday. On AutomationBench, Mistral Large 4 took on 657 business workflows across simulated workplace apps, all four included, finish

→8 Oct 2026Quad-State Safety Evaluation of Open-Weight Large Language Models on Non-Canonical Inputs

arXiv:2610.09033v1 Announce Type: new Abstract: Standard safety evaluations of large language models assess harmful requests written in canonical plain text, while models in real-world deployment rout

→8 Oct 2026How Do Agentic LLMs Decide to Call Tools? A Tool-Call Vector Shaped by Suppression

arXiv:2610.09624v1 Announce Type: cross Abstract: Tool calling, invoking external tools on demand, is central to agentic LLMs, yet the mechanism that decides whether a model calls a tool or responds d

→8 Oct 2026Document-Level Text Simplification in Estonian Using Large Language Models

arXiv:2610.10378v1 Announce Type: new Abstract: Document-level text simplification involves transformations that go beyond sentence-internal edits, addressing discourse coherence, anaphora resolution,

→9 Oct 2026Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL

arXiv:2607.16204v2 Announce Type: replace Abstract: Recent growth in reinforcement learning (RL) has surfaced a need for diverse, specialized training environments. Hand-curated environments with fixe

→11 Oct 2026Every Model That Can Be Run On 10-16GB VRAM Ranked

It's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features. Fortunately, last year brought new archit

→12 Oct 2026CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation

arXiv:2610.13257v1 Announce Type: cross Abstract: Cross-tokenizer on-policy distillation (OPD) requires aligning teacher and student sequences before their scores can be compared. Decoded-text alignme

CompanyxAI8 recent entries
7 Oct 2026Rule-Based Languages for Neurosymbolic AI

arXiv:2610.07313v1 Announce Type: new Abstract: Logic programming is increasingly used as the symbolic component of neurosymbolic AI systems. We survey the main rule-based languages in this setting, n

→7 Oct 2026Generalizable Robustness Testing of DNN-Based Robotic Navigation Systems via XAI-Guided Search

arXiv:2610.06862v1 Announce Type: new Abstract: **Context:** Deep Neural Networks (DNNs) increasingly control Cyber-Physical Systems (CPSs), yet small input perturbations can cause unsafe system-level

→7 Oct 2026FlowCF: Sparse Counterfactual Explanations for Mixed-Type Tabular Data using Flow Matching

arXiv:2610.08537v1 Announce Type: cross Abstract: In the field of Explainable AI (XAI), counterfactual (CF) explanations interpret a model's decision by suggesting the changes to the input that would

→7 Oct 2026CCDF: A Benchmark Dataset for Deepfake Detection in Real-World Surveillance Footage

arXiv:2610.07939v1 Announce Type: new Abstract: Due to rapid advances in Generative AI, commercial video generation tools can be used to produce fabricated surveillance footage that can fool both huma

→8 Oct 2026Jevjitsu: Or, How We Tried Generalized Classifiers on Everything

Classification is one of the oldest problems in machine learning. Unlike modern decoding transformers, most classification models were non-autoregressive by nature. Before Jev, the common approaches w

→9 Oct 2026WorldBench: Evaluating LLMs on Three.js Voxel World Generation

arXiv:2610.10622v1 Announce Type: cross Abstract: Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges tak

→12 Oct 2026People Making Conversation: Evaluating How Naturally Speech Models Converse

arXiv:2610.12802v1 Announce Type: new Abstract: A conversational speech model has to decide, at every pause, whether to speak. Existing evaluations test boundary detection, policy adherence, or aggreg

→12 Oct 2026Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation

arXiv:2610.12703v1 Announce Type: cross Abstract: High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation a

CompanyDeepSeek8 recent entries
9 Oct 2026Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild

arXiv:2610.11328v1 Announce Type: new Abstract: Advances in foundation models are driving efforts to introduce agents to assist people in the physical world. Such agents require agentic spatial intell

→9 Oct 2026Just another purely open-weight models benchmark

Live results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule area

→9 Oct 2026Evaluating Rubric Generation with Interventional Transfer

arXiv:2610.10809v1 Announce Type: new Abstract: Instance-specific rubrics are common in AI benchmarks where reliable evaluation requires specific expert knowledge. This approach is difficult to scale,

→9 Oct 2026Bullish on this trend of making post-training more accessible. A new post-training era is upon us. If you work on agentic RL, long-context t…

Bullish on this trend of making post-training more accessible. A new post-training era is upon us. If you work on agentic RL, long-context tasks (a big focus today) are expensive, inefficient, and don

→9 Oct 2026A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimization

arXiv:2610.12183v1 Announce Type: cross Abstract: Black-box optimization (BBO) arises in many scientific and engineering problems where objective evaluations are expensive and limited. Recent large la

→11 Oct 2026Every Model That Can Be Run On 10-16GB VRAM Ranked

It's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features. Fortunately, last year brought new archit

→12 Oct 2026Not All Correct Fixes Run Alike: An Empirical Study of Patch Performance in Repository-Level Issue Resolution

arXiv:2610.13563v1 Announce Type: cross Abstract: Repository-level issue-resolution benchmarks such as SWE-bench evaluate submitted patches primarily based on whether they pass the correctness tests,

→12 Oct 2026Let Large Language Models Judge Each Other: Multi-Agent Peer-Reviewed Reasoning for Medical Question Answering

arXiv:2606.15419v2 Announce Type: replace-cross Abstract: Objective: To enhance the accuracy, interpretability, and robustness of large language models (LLMs) in medical question answering (MedQA). Me

CompanyNVIDIA8 recent entries
9 Oct 2026Traceable World State: A Provenance-Aware State Representation and Deterministic Replay Framework for Robotic Systems

arXiv:2610.12033v1 Announce Type: cross Abstract: Robotic systems operating over extended tasks must maintain a world state assembled from observations arriving at different times, with varying confid

→9 Oct 2026Super interesting NVIDIA paper on choosing base models for coding agents. It's actually a clever way to rank base checkpoints by how well ea…

Super interesting NVIDIA paper on choosing base models for coding agents. It's actually a clever way to rank base checkpoints by how well each one is likely to do as a coding agent after post-training

→9 Oct 2026SatFix: Absolute Visual Localization of UAVs in Satellite Maps from a Single Oblique Image

arXiv:2610.11049v1 Announce Type: new Abstract: We study absolute metric UAV localization within a provided geo-referenced satellite region, recovering continuous map position and viewing heading from

→9 Oct 2026Just another purely open-weight models benchmark

Live results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule area

→9 Oct 2026GLIO2: A GPU-Parallelized Tightly-Coupled LiDAR-Inertial-GNSS System for Robust and Real-Time Global Localization and Mapping

arXiv:2610.12411v1 Announce Type: new Abstract: Globally consistent, real-time state estimation in large-scale, perceptually degraded environments is essential for autonomous vehicles and aerial robot

→9 Oct 2026Demonstrating Arena 5.0: A Photorealistic ROS2 Simulation Framework for Developing and Benchmarking Social Navigation

arXiv:2610.11220v1 Announce Type: new Abstract: Building upon the foundations laid by our previous work, this paper introduces Arena 5.0, the fifth iteration of our framework for robotics social navig

→11 Oct 2026I trained a 102M recursive BitNet-v2 model from scratch: 64K context, trained on less than 5B tokens

DISCLAIMER: the post and the model card was made with the assist of AI. Hiya, I’m releasing Recursive BitNet N-Gram 102M, a small experiment combining ternary weights, shared transformer layers, and h

→11 Oct 2026Every Model That Can Be Run On 10-16GB VRAM Ranked

It's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features. Fortunately, last year brought new archit