AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries123,303
  • Agents10,352
  • Applications7,013
  • Companies3
  • Concepts7
  • Hardware2,702
  • Industry6,776
  • Local Ai6,847
  • Model Releases35,469
  • Research28,731
  • Safety18,972
  • Syntheses30
  • Tools1,775
  • Tutorials4,626

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries123,303
  • Agents10,352
  • Applications7,013
  • Companies3
  • Concepts7
  • Hardware2,702
  • Industry6,776
  • Local Ai6,847
  • Model Releases35,469
  • Research28,731
  • Safety18,972
  • Syntheses30
123,303Total entries
59Added by human
123,244Found by agent
13Categories

Knowledge catalogue

Search: “evaluation”

GridTimelineEvolution
39,434 results
CompaniesToolsTechniques

Each lane shows up to 8 recent matching entries, ordered from earlier to later. Tracks load separately to keep the 75,000+ entry wiki fast.

Companies

CompanyAnthropic8 recent entries
12 Oct 2026Masked Diffusion Language Agents for Tool-Integrated Chemical Reasoning

arXiv:2610.13330v1 Announce Type: new Abstract: Designing tool-integrated reasoning agent systems for complex chemical tasks remains a fundamental challenge. Although recent chemical agents have demon

→12 Oct 2026LongHarness: Advancing Agentic Reasoning over 100M-Token Corpora via Stateful Evidence Management

arXiv:2610.13075v1 Announce Type: new Abstract: Corpus-level long-context reasoning requires agents to acquire and integrate evidence across document collections that exceed their context windows. Exi

Tools
1,775
  • Tutorials4,626

  • Source
    HumanDGX agent

    Content type
    AllBlogX PostPaperYouTubeRedditGitHub
    Clear filters
    →12 Oct 2026Introducing DigitalOcean Signals and Simulations: see how your agents and models really behave in production, then test the fix before it ships

    You ship an agent. It mostly works. Then the feedback loop becomes a support ticket: a user got the wrong answer, a tool call looped four times, a refund went out that shouldn’t have. Standard observa

    →12 Oct 2026EvoBelief: Falsifiable World Models for Self-Improving LLM Agents

    arXiv:2610.12789v1 Announce Type: new Abstract: Self-evolving LLM agents typically improve how they act through accumulated experience, reflection, or skills. Yet even procedurally capable agents can

    →12 Oct 2026Anthropic researchers say small AI models may be months away from being able to self-replicate on IoT devices like smart ovens and off-brand Android TV boxes (Anthropic)

    Anthropic: Anthropic researchers say small AI models may be months away from being able to self-replicate on IoT devices like smart ovens and off-brand Android TV boxes — Our experiments suggest that

    →13 Oct 2026Sources: Anthropic and OpenAI have pushed for substantial control over what external AI evaluators can publish, out of concern that trade secrets could leak (Aditya Soni/Reuters)

    Aditya Soni / Reuters: Sources: Anthropic and OpenAI have pushed for substantial control over what external AI evaluators can publish, out of concern that trade secrets could leak — Anthropic and Open

    →13 Oct 2026One Agent Development Kit, five languages

    When we talk about agents, that doesn't necessarily mean we're talking about Python. In today's modern ecosystem, an agent might run as a high-concurrency Go microservice on Cloud Run, or live inside

    →13 Oct 2026DigitalOcean Inference Router Enhancements: Performance Tuned Routing, No Rules to Write

    Every few weeks, a new model steps up and claims best-in-class at something. The launch chart makes the case, the link makes the rounds, and your team is back to the question nobody has time for: shou

    CompanyOpenAI8 recent entries
    9 Oct 2026Large Language Model Turnover Undermines Screening for Artificial Intelligence-Assisted Scientific Writing

    arXiv:2610.11599v1 Announce Type: cross Abstract: Journals and conferences have begun to screen submitted manuscripts for text written using large language models (LLMs). The reliability of this scree

    →9 Oct 2026From Reactive Containment to Proactive Assurance: Lessons from OpenAI, Anthropic, and Google Agent Security Incidents

    arXiv:2610.12463v1 Announce Type: cross Abstract: In 2026, cybersecurity evaluations involving OpenAI, Anthropic, and Google agents reached real systems outside their authorized test scope. The paths

    →9 Oct 2026Arbiter: Detecting Interference in LLM Agent System Prompts

    arXiv:2603.08993v2 Announce Type: replace-cross Abstract: System prompts for LLM-based coding agents are software artifacts that govern agent behavior, yet lack the testing infrastructure applied to c

    →10 Oct 2026Engineer / developer observations of Gemma4-31B, Qwen3.8-27B, and 6.1-Sol for software engineering work

    For what it's worth. I'm a mechanical engineer by education and software engineer in practice over the past ~20 years. Playing around with local models has been a recent hobby. Today I decided to do a

    →10 Oct 2026A look at a 1,700-member Slack run by Medicare agency CMS where Microsoft, OpenAI, and other companies help shape policy on AI apps and medical records access (CBS News)

    CBS News: A look at a 1,700-member Slack run by Medicare agency CMS where Microsoft, OpenAI, and other companies help shape policy on AI apps and medical records access — A cadre of elite techies, fin

    →13 Oct 2026Sources: Anthropic and OpenAI have pushed for substantial control over what external AI evaluators can publish, out of concern that trade secrets could leak (Aditya Soni/Reuters)

    Aditya Soni / Reuters: Sources: Anthropic and OpenAI have pushed for substantial control over what external AI evaluators can publish, out of concern that trade secrets could leak — Anthropic and Open

    →13 Oct 2026One Agent Development Kit, five languages

    When we talk about agents, that doesn't necessarily mean we're talking about Python. In today's modern ecosystem, an agent might run as a high-concurrency Go microservice on Cloud Run, or live inside

    →13 Oct 2026DigitalOcean Inference Router Enhancements: Performance Tuned Routing, No Rules to Write

    Every few weeks, a new model steps up and claims best-in-class at something. The launch chart makes the case, the link makes the rounds, and your team is back to the question nobody has time for: shou

    CompanyGoogle8 recent entries
    12 Oct 2026People Making Conversation: Evaluating How Naturally Speech Models Converse

    arXiv:2610.12802v1 Announce Type: new Abstract: A conversational speech model has to decide, at every pause, whether to speak. Existing evaluations test boundary detection, policy adherence, or aggreg

    →12 Oct 2026MathVis: Can AI Reason through Drawing? A Benchmark for Mathematical Figure Generation

    arXiv:2610.12954v1 Announce Type: new Abstract: This paper introduces MathVis, a benchmark designed to evaluate LLMs' ability to visualize frontier mathematical problems and perform diagram-assisted r

    →12 Oct 2026Introducing DigitalOcean Signals and Simulations: see how your agents and models really behave in production, then test the fix before it ships

    You ship an agent. It mostly works. Then the feedback loop becomes a support ticket: a user got the wrong answer, a tool call looped four times, a refund went out that shouldn’t have. Standard observa

    →12 Oct 2026APEX-Harm: Evaluating Agent Safety in Shared Professional Workspaces

    arXiv:2610.13623v1 Announce Type: cross Abstract: Professional agents increasingly work through command-line interfaces and multiple tools over shared workspaces containing files, messages, and busine

    →12 Oct 2026AdaC-GRPO: Towards Mastering Long-Horizon Visual Sudoku Puzzles via Adaptive Curriculum Learning

    arXiv:2610.13184v1 Announce Type: new Abstract: While reinforcement learning with verifiable reward (RLVR) enhances reasoning in large language models, its efficacy in long-horizon multimodal tasks re

    →13 Oct 2026One Agent Development Kit, five languages

    When we talk about agents, that doesn't necessarily mean we're talking about Python. In today's modern ecosystem, an agent might run as a high-concurrency Go microservice on Cloud Run, or live inside

    →13 Oct 2026Gemini 4 Argon available on Google Cloud, built to help solve your business’ hardest problems

    Last week we announced that Gemini Enterprise has evolved to make it even simpler to get work done, and introduced the Gemini agent, your new universal agent for work. It's always on and ready to answ

    →14 Oct 2026Benchmarks are becoming the core competency of any company. Our company mission is to build the world’s best models for document processing.…

    Benchmarks are becoming the core competency of any company. Our company mission is to build the world’s best models for document processing. This also means that we have to build the world’s best benc

    CompanyMeta8 recent entries
    12 Oct 2026Hyper Algorithm Design Agent: Evolving Learnable Optimizer from Zero

    arXiv:2609.35328v2 Announce Type: replace Abstract: Meta-Black-Box Optimization (MetaBBO) is one of the highlights in the recent AI for Optimization trend. This paradigm's bi-level workflow leverages

    →12 Oct 2026DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs

    arXiv:2610.13767v1 Announce Type: cross Abstract: Low-bit inference on accelerators with native 4-bit support requires quantizing both weights and activations to realize the full throughput gains thes

    →12 Oct 2026CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation

    arXiv:2610.13257v1 Announce Type: cross Abstract: Cross-tokenizer on-policy distillation (OPD) requires aligning teacher and student sequences before their scores can be compared. Decoded-text alignme

    →12 Oct 2026Byzantine-Resilient Federated Large Language Model via Hypergraph Message Passing

    arXiv:2610.12711v1 Announce Type: new Abstract: Federated Large Language Models (FedLLMs) are promising to deliver privacy-preserving, collaborative intelligence. The decentralized training process is

    →12 Oct 2026Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs

    arXiv:2610.13078v1 Announce Type: new Abstract: Safety-aligned large language models (LLMs) remain vulnerable to jailbreaks and can over-refuse benign requests. Understanding these failures requires d

    →12 Oct 2026Agreement Without Alignment: Label-Free Rubric Clarification Increases LLM-Judge Consensus but Not Agreement with Humans

    arXiv:2610.13303v1 Announce Type: cross Abstract: LLM judges are increasingly deployed as panels, and disagreement among panel members is often read as a symptom of an unclear rubric. We test a label-

    →13 Oct 2026Update: Converting dense models into Mixture-of-Experts

    This is an update to my previous post which can be found here: https://www.reddit.com/r/LocalLLaMA/s/n7sgRZZc4z Since my last post I have continued working on the conversion process and also convertin

    →13 Oct 2026Inside Action Gateway in Managed Agents

    A coding agent finishes a change and wants to open a GitHub issue with a link to its results. The agent can describe the issue, but creating it belongs to a different system. It needs a tool that can

    CompanyMistral8 recent entries
    8 Oct 2026Quad-State Safety Evaluation of Open-Weight Large Language Models on Non-Canonical Inputs

    arXiv:2610.09033v1 Announce Type: new Abstract: Standard safety evaluations of large language models assess harmful requests written in canonical plain text, while models in real-world deployment rout

    →8 Oct 2026How Do Agentic LLMs Decide to Call Tools? A Tool-Call Vector Shaped by Suppression

    arXiv:2610.09624v1 Announce Type: cross Abstract: Tool calling, invoking external tools on demand, is central to agentic LLMs, yet the mechanism that decides whether a model calls a tool or responds d

    →8 Oct 2026Document-Level Text Simplification in Estonian Using Large Language Models

    arXiv:2610.10378v1 Announce Type: new Abstract: Document-level text simplification involves transformations that go beyond sentence-internal edits, addressing discourse coherence, anaphora resolution,

    →9 Oct 2026Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL

    arXiv:2607.16204v2 Announce Type: replace Abstract: Recent growth in reinforcement learning (RL) has surfaced a need for diverse, specialized training environments. Hand-curated environments with fixe

    →11 Oct 2026Every Model That Can Be Run On 10-16GB VRAM Ranked

    It's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features. Fortunately, last year brought new archit

    →12 Oct 2026🇫🇷 Le Chonk joins the sudo L7 leaderboard 🇫🇷 We just added @MistralAI's Mistral Large 4 to sudo L7, our benchmark for staff-level softwa…

    🇫🇷 Le Chonk joins the sudo L7 leaderboard 🇫🇷 We just added @MistralAI's Mistral Large 4 to sudo L7, our benchmark for staff-level software engineering. Mistral Large 4 scores 13.3%, ahead of Muse Spar

    →12 Oct 2026CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation

    arXiv:2610.13257v1 Announce Type: cross Abstract: Cross-tokenizer on-policy distillation (OPD) requires aligning teacher and student sequences before their scores can be compared. Decoded-text alignme

    →14 Oct 2026Benchmarks are becoming the core competency of any company. Our company mission is to build the world’s best models for document processing.…

    Benchmarks are becoming the core competency of any company. Our company mission is to build the world’s best models for document processing. This also means that we have to build the world’s best benc

    CompanyxAI8 recent entries
    7 Oct 2026FlowCF: Sparse Counterfactual Explanations for Mixed-Type Tabular Data using Flow Matching

    arXiv:2610.08537v1 Announce Type: cross Abstract: In the field of Explainable AI (XAI), counterfactual (CF) explanations interpret a model's decision by suggesting the changes to the input that would

    →7 Oct 2026CCDF: A Benchmark Dataset for Deepfake Detection in Real-World Surveillance Footage

    arXiv:2610.07939v1 Announce Type: new Abstract: Due to rapid advances in Generative AI, commercial video generation tools can be used to produce fabricated surveillance footage that can fool both huma

    →8 Oct 2026Jevjitsu: Or, How We Tried Generalized Classifiers on Everything

    Classification is one of the oldest problems in machine learning. Unlike modern decoding transformers, most classification models were non-autoregressive by nature. Before Jev, the common approaches w

    →9 Oct 2026WorldBench: Evaluating LLMs on Three.js Voxel World Generation

    arXiv:2610.10622v1 Announce Type: cross Abstract: Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges tak

    →12 Oct 2026People Making Conversation: Evaluating How Naturally Speech Models Converse

    arXiv:2610.12802v1 Announce Type: new Abstract: A conversational speech model has to decide, at every pause, whether to speak. Existing evaluations test boundary detection, policy adherence, or aggreg

    →12 Oct 2026🇫🇷 Le Chonk joins the sudo L7 leaderboard 🇫🇷 We just added @MistralAI's Mistral Large 4 to sudo L7, our benchmark for staff-level softwa…

    🇫🇷 Le Chonk joins the sudo L7 leaderboard 🇫🇷 We just added @MistralAI's Mistral Large 4 to sudo L7, our benchmark for staff-level software engineering. Mistral Large 4 scores 13.3%, ahead of Muse Spar

    →12 Oct 2026Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation

    arXiv:2610.12703v1 Announce Type: cross Abstract: High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation a

    →13 Oct 2026Build long-horizon agents with Grok 4.7 on Amazon Bedrock

    Grok 4.7 is now available on Amazon Bedrock. In this post, we build a tool-calling agent on Grok 4.7 and evaluate it on three reasoning tasks at four reasoning effort levels, measuring correctness and

    CompanyDeepSeek8 recent entries
    9 Oct 2026Just another purely open-weight models benchmark

    Live results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule area

    →9 Oct 2026Evaluating Rubric Generation with Interventional Transfer

    arXiv:2610.10809v1 Announce Type: new Abstract: Instance-specific rubrics are common in AI benchmarks where reliable evaluation requires specific expert knowledge. This approach is difficult to scale,

    →9 Oct 2026Bullish on this trend of making post-training more accessible. A new post-training era is upon us. If you work on agentic RL, long-context t…

    Bullish on this trend of making post-training more accessible. A new post-training era is upon us. If you work on agentic RL, long-context tasks (a big focus today) are expensive, inefficient, and don

    →9 Oct 2026A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimization

    arXiv:2610.12183v1 Announce Type: cross Abstract: Black-box optimization (BBO) arises in many scientific and engineering problems where objective evaluations are expensive and limited. Recent large la

    →11 Oct 2026Every Model That Can Be Run On 10-16GB VRAM Ranked

    It's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features. Fortunately, last year brought new archit

    →12 Oct 2026Not All Correct Fixes Run Alike: An Empirical Study of Patch Performance in Repository-Level Issue Resolution

    arXiv:2610.13563v1 Announce Type: cross Abstract: Repository-level issue-resolution benchmarks such as SWE-bench evaluate submitted patches primarily based on whether they pass the correctness tests,

    →12 Oct 2026Let Large Language Models Judge Each Other: Multi-Agent Peer-Reviewed Reasoning for Medical Question Answering

    arXiv:2606.15419v2 Announce Type: replace-cross Abstract: Objective: To enhance the accuracy, interpretability, and robustness of large language models (LLMs) in medical question answering (MedQA). Me

    →13 Oct 2026DigitalOcean Inference Router Enhancements: Performance Tuned Routing, No Rules to Write

    Every few weeks, a new model steps up and claims best-in-class at something. The launch chart makes the case, the link makes the rounds, and your team is back to the question nobody has time for: shou

    CompanyNVIDIA8 recent entries
    9 Oct 2026Super interesting NVIDIA paper on choosing base models for coding agents. It's actually a clever way to rank base checkpoints by how well ea…

    Super interesting NVIDIA paper on choosing base models for coding agents. It's actually a clever way to rank base checkpoints by how well each one is likely to do as a coding agent after post-training

    →9 Oct 2026SatFix: Absolute Visual Localization of UAVs in Satellite Maps from a Single Oblique Image

    arXiv:2610.11049v1 Announce Type: new Abstract: We study absolute metric UAV localization within a provided geo-referenced satellite region, recovering continuous map position and viewing heading from

    →9 Oct 2026Just another purely open-weight models benchmark

    Live results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule area

    →9 Oct 2026GLIO2: A GPU-Parallelized Tightly-Coupled LiDAR-Inertial-GNSS System for Robust and Real-Time Global Localization and Mapping

    arXiv:2610.12411v1 Announce Type: new Abstract: Globally consistent, real-time state estimation in large-scale, perceptually degraded environments is essential for autonomous vehicles and aerial robot

    →9 Oct 2026Demonstrating Arena 5.0: A Photorealistic ROS2 Simulation Framework for Developing and Benchmarking Social Navigation

    arXiv:2610.11220v1 Announce Type: new Abstract: Building upon the foundations laid by our previous work, this paper introduces Arena 5.0, the fifth iteration of our framework for robotics social navig

    →11 Oct 2026I trained a 102M recursive BitNet-v2 model from scratch: 64K context, trained on less than 5B tokens

    DISCLAIMER: the post and the model card was made with the assist of AI. Hiya, I’m releasing Recursive BitNet N-Gram 102M, a small experiment combining ternary weights, shared transformer layers, and h

    →11 Oct 2026Every Model That Can Be Run On 10-16GB VRAM Ranked

    It's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features. Fortunately, last year brought new archit

    →12 Oct 2026ReplaySSM: Rethinking State Storage for Faster Hybrid-Model Decoding

    ReplaySSM stores an SSM state as a high‑precision checkpoint plus a buffer of recent inputs, cutting memory traffic by ~50 % and accelerating SSM decoding kernels on NVIDIA Blackwell Ultra B300 GPUs b