Model Releases
5 Frontier LLM's grade 39 local LLM's on 50 prompts
This is just something i had running for about 10-20 hours (kept adding models), various tests/benchmarks and had 4 (Claude kind of graded, his tokens are too precious) frontier LLM's grade the answer
This is just something i had running for about 10-20 hours (kept adding models), various tests/benchmarks and had 4 (Claude kind of graded, his tokens are too precious) frontier LLM's grade the answers of the local LLM's. Big models got about 44 and smaller models got 66 prompt/tests. Some written by me, most by AI, most Ai generated tests based off work ive done with ai agents. Grading took about $30-40 for argon and 2x5 hour limits for codex grading and who cares about antigravity Gemini 3.8. Astra, sol, terra, argon, Gemini and opus 5.5 all wrote some tests, we all per capita probably wrote about the same amount of test/benchmarks All tested on a 5060 ti 16GB, 64GB ram, 5800x3d Outputs are 3k tokens and some tests like debugging only had 1 prompt, for debugging specifically it was a real bug I came across in a project After grading, each of the four judges looked at its own score table, agent counts, speeds and six sample answers, and named two models: its favorite (most capable today) and the best foundation (the one to fine-tune and give specialist models). Gemini 3.8 Top 3 capability: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Qwen3.6 35B-A3B Top 3 foundation: Nanbeige4.2 3B, Gemma-4 12B Q6, Bonsai-2 27B PQ2 Terra (GPT-6.1) Top 3 capability: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Gemma-4 12B Q6 Top 3 foundation: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Nanbeige4.2 3B Sol (GPT-6.1 Sol) Top 3 capability: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Qwen3.6 35B-A3B Top 3 foundation: Underdog-Saluki 27B, Bonsai-2 27B PQ2, Qwen3.6 35B-A3B Argon (Gemini 4) Top 3 capability: Underdog-Saluki 27B, Bonsai-2 27B PQ2, Gemma-4 12B Q6 Top 3 foundation: Gemma-4 12B Q6, Qwen3.6 35B-A3B, OmniCoder 9B Q8 submitted by /u/yobarisushcatel [link] [comments]
Related
- Anybody use frontier models like Astra/Fable for planning/judging, and qwen3.8 as the main workhorse? Curious to hear about your setups!
- I benchmarked Qwen 3.8 27B on my M4 Max — here's what I learned about local models, context, and coding performance
- I built a local GPU benchmark server for 4B to 35B models because I was tired of synthetic cloud leaderboards and tiny sample tests
- Two local Qwen ( 3.8 27b unsloth Q6 and Qwen flash next strata coder ) models vs Claude Opus 4.6 on the same 3 coding tasks. One of them tied it. Not here to start a fight, just sharing numbers
Source: r/LocalLLaMA | 2026-10-29