Model Releases
Qwen3.8-27B scored 29/30 on AIME 2026 with FP8 + xhigh reasoning — BF16 vs FP8 results
I benchmarked Qwen3.8-27B on MathArena/aime_2026 dataset, comparing BF16 and FP8 weights at medium and xhigh reasoning effort. Interesting findings are: quantized FP8 xhigh is better than BF 16 medium
I benchmarked Qwen3.8-27B on MathArena/aime_2026 dataset, comparing BF16 and FP8 weights at medium and xhigh reasoning effort. Interesting findings are: quantized FP8 xhigh is better than BF 16 medium equally good as 16 BF xhigh with better speed. On problem 7, both BF16 xhigh and quantized FP8 xhigh exhausted the full context token generation budget without producing a final answer. So those were empty not wrong. Final results Model · thinking Score Max tokens Pre-fill tk/s Decode tk/s BF16 · medium 28/30 (93.3%) 58,747 1.9K 28 BF16 · xhigh 29/30 (96.7%) 258,048 1.9 28 Quantized FP8 · medium 26/30 (86.7%) 75,024 3.4K 76 Quantized FP8 · xhigh 29/30 (96.7%) 258,048 3.4K 76 Evaluation settings Dataset: MathArena/aime_2026, Dataset revision: d2de22f3c656b4f56cf8981212186377d1e23bc3 Exact-match scoring Sampling: disabled BF16 concurrency: 4 FP8 concurrency: 7 Same chat template and prompt format across all runs Temperature: zero How it compares with frontier models Note : looks like others ran it multiple times on their models and averaged, I ran it only once. Model and setting Reported AIME 2026 score GPT-5.6 Sol, xhigh 99.9%* GLM-5.2 99.2% GPT-5.4, xhigh 99.2% Gemini 3.1 Pro, high 98.3% Inkling, effort 0.99 97.1% Claude Opus 4.6, max 96.7% DeepSeek V4 Pro 96.7% Qwen3.8-27B FP8, xhigh — our pass@1 run 96.7% (29/30) Kimi K2.6, thinking 96.4% Qwen3.6-27B 94.1% submitted by /u/No_Run8812 [link] [comments]
Source: r/LocalLLaMA | 2026-08-20