Agents
jev as a Judge proves to be a cheaper and more precise alternative to LLM as a judge for online evals. great guide from Sean and Daniel on w…
jev as a Judge proves to be a cheaper and more precise alternative to LLM as a judge for online evals. great guide from Sean and Daniel on why Jev is great for evals, an experiment vs other LLMs, and
jev as a Judge proves to be a cheaper and more precise alternative to LLM as a judge for online evals. great guide from Sean and Daniel on why Jev is great for evals, an experiment vs other LLMs, and how to try this out for your agents! We tested Jev against LLM judges on accuracy, repeatability, latency, and cost to see whether System One models could offer a new approach to agent evaluation. https://x.com/i/article/2101448785255907328
Related
- great write up, evals are hard! here are 2 broad buckets we use to evaluate agents: 1. Measure the State of the World 2. Agent as a Judge on…
- Verifiers are important for scaling evals/RL But costs add up! So can we make them cheaper? Some great work by @Vtrivedy10 @jakebroekhuizen …
- Agents will cheat your evals if you let them. @Vtrivedy10 wrote a great article on how we keep them from overfitting
Source: Harrison Chase (X) | 2026-09-20