VeriLoop E2 GGUF Release: BF16 → IQ1_M, 50.11 GiB → 16.79 GiB with +0.319% PPL drift

We’ve released the complete VeriLoop E2 GGUF precision ladder, from the canonical BF16 reference down to a mixed-precision IQ1_M deployment tier. This post is specifically about the GGUF release and local deployment. The parent VeriLoop E2 model was introduced separately. The goal was not simply to generate as many quantizations as possible. We wanted a measured deployment ladder where every tier is evaluated against the same frozen BF16 reference, and where the low-footprint variants retain a deliberately protected higher-precision spine instead of uniformly pushing every tensor to the lowest available bit width. The practical recommendation is: Q6_K if you want the strongest overall quality / memory balance. Q5_K_M if you need to get below ~19 GiB while retaining wider fidelity margins. IQ1_M if minimum model-file footprint is the priority. IQ2_S if you want essentially the same footprint as IQ1_M but slightly lower Mean KLD. Q8_0 / BF16 when fidelity matters more than local-memory cost. A Full List of GGUF Precision Ladder: Tier Main size Reduction vs BF16 PPL ratio Mean KLD Same top-p Recommended role BF16 50.113 GiB — 1.000000 reference 100% Canonical reference Q8_0 26.632 GiB 46.86% 1.000643 0.002176 98.815% High fidelity Q6_K 20.566 GiB 58.96% 0.999605 0.004409 98.204% Overall sweet spot Q5_K_M 18.965 GiB 62.16% 1.004450 0.006919 97.251% Memory-quality sweet spot Q4_K_M 18.301 GiB 63.48% 1.004821 0.009700 96.786% Balanced compact Q3_K_M 16.826 GiB 66.42% 1.004090 0.014349 95.919% Low-footprint alternative IQ2_S 16.799 GiB 66.48% 1.003457 0.014023 95.516% Lower-KLD low-footprint option IQ1_M 16.790 GiB 66.50% 1.003191 0.014357 95.870% Minimum-footprint sweet spot submitted by /u/PleaseLee [link] [comments]

VeriLoop E2 is now open on Hugging Face — curious what Ollama/GGUF quantization would make the most sense for this 27B model

Source: r/ollama | 2026-09-25

Related