TechniqueRLHF / Alignment2 recent entries22 Jul 2026How to measure human-LLM judge alignmentNo single metric proves an LLM judge is trustworthy. This field guide shows how to measure human–human agreement, compare it to LLM–human agreement, and diagnose errors with precision, recall, and F1.→9 Oct 2026Alignment is a human-AI workflow: how Custom Views make agent review possibleAutomated evals can score every agent session, but they do not automatically reflect what your team considers a good outcome. That takes human judgment. The post Alignment is a human-AI workflow: how
TechniqueFine-tuning4 recent entries27 May 2026From production traces to better AI agents: Automating the LLMOps feedback loopProduction AI traces are the raw material for better evals, prompts, datasets, and fine-tuned models. This post shows how the Arize AX Airflow Provider turns that feedback loop into scheduled, monitor→2 Jun 2026The end of fine-tuning: Why evals, context, and traces matter moreFine-tuning isn't dead, but the way most teams iterate on AI products has split in two. A tiny fraction run continuous RL against their own environments; everyone else has moved the iteration loop out→13 Jul 2026How do you make an LLM, anyway? Microsoft just published a textbook.Microsoft published a 109-page technical report on MAI-Thinking-1. Here’s the abbreviated version of how a modern lab actually trains a frontier reasoning model — from scraping the web to reinforcemen→18 Sept 2026TypeSafe’s Jev: Can decision models replace LLM judges?TypeSafe’s Jev classifies, scores, and routes without generating text — up to hundreds of times cheaper than an LLM judge. What that changes for evals, confidence routing, and application architecture
TechniqueMultimodal2 recent entries4 Jun 2026Building the AI factory for self-improving agents: What’s new in Arize AXArize AX is adding managed agents, full-agent experimentation, expanded multimodal support, and Harness-as-a-Judge to help teams observe, evaluate, and improve production agents. The post Building the→2 Oct 2026Prompt caching benchmark: high cache reuse doesn’t always mean lower costWe benchmarked prompt caching across DeepSeek, GLM, GPT, and Claude using Harbor evals and Phoenix traces, so we could compare cache reuse, estimated cost, and latency on the same multi-turn shopping
TechniqueSafety8 recent entries13 Aug 2026AI agent guardrails vs. evals: How to build more reliable agent systemsGuardrails constrain what an agent can do in code; evals judge whether it performed well. Learn how both layers—and the harness around them—make long-running AI agents reliable. The post AI agent guar→19 Aug 2026Where agent evals are going: Agent-as-a-JudgeAgents changed what failure looks like, and the evaluation layer has to change with them. Why agent-as-a-judge is moving from research paper to production eval stack. The post Where agent evals are go→2 Sept 2026How Coinbase Wallet built an agent-first product development lifecycleBy redesigning planning, validation, and risk review around AI agents, Coinbase Wallet dramatically shortened the path from product idea to working software. The post How Coinbase Wallet built an agen→14 Sept 2026AI agent regression testing with Agent Experiments in Arize AXA cancellation-policy fix raised action safety and dropped average task completion from 0.89 to 0.72. This walkthrough shows how to regression-test agent changes with Agent Experiments in Arize AX. Th→23 Sept 2026Real-time LLM guardrails with Jev: comparing latency and costCompare Jev and GPT-5.4 nano for real-time LLM guardrails, with demo results on latency, cost, and checks on agent inputs, replies, and tool calls. The post Real-time LLM guardrails with Jev: comparin→30 Sept 2026NVIDIA proposes an AI agent kill switch in silicon after a year of sandbox escapesThis year agents at OpenAI, Anthropic, and Google escaped environments meant to contain them. What decided severity was time-to-detect and time-to-kill, from 12 minutes to seven months. NVIDIA's Open →8 Oct 2026Cost Agent: find where your LLM and agent spend goes with your tracesAI costs don’t always scale predictably with traffic. A retry loop, oversized context window, or unnecessarily expensive model can make a small number of traces disproportionately expensive. The post →9 Oct 2026Alignment is a human-AI workflow: how Custom Views make agent review possibleAutomated evals can score every agent session, but they do not automatically reflect what your team considers a good outcome. That takes human judgment. The post Alignment is a human-AI workflow: how