[Release] WinterMix — Qwen3.5-122B-A10B in native MLX: an 82 GiB build that beats 94–95 GiB quants, plus a 68 GiB build for agent swarms

TL;DR: I spent 9 days developing a new quantization method for MLX models and measured 18 variants against each other on a single M5 Max MacBook Pro (128 GB). The result is the best-measuring MLX quant of Qwen3.5-122B-A10B I'm aware of at any size — the 82 GiB build edges out 94–95 GiB 6-bit builds, and lands within 0.3–0.7% of the imatrix-rounded source GGUF while staying native MLX. Apache 2.0, weights up on HF. Why bother if GGUF is better? MLX on Apple Silicon is substantially faster than llama.cpp on the same hardware — on my M5 Max I measure roughly 9x faster prefill and ~20% faster token generation. For anything with a long context and a lot of turns, that gap compounds. The problem is that existing MLX quants below 6 bit are not great, and you can see it in the table below: oQ4 gives up ~3.8% perplexity at short context and ~4.2% at long context against the source GGUF. In practice that shows up as incoherent reasoning traces and rounding errors that stack until the model starts hallucinating. So a better MLX quantization method has real advantages for agentic workflows and local AI on Apple Silicon. At the same time, I made the conscious decision to require native MLX support. imatrix on MLX is not format native — it needs custom kernels. WinterMix quants are format native and are drop-in replacements. WinterMix quantized models are format-native MLX models with open weights (Apache 2.0). No custom kernels, no forked runtime, no flags. They load anywhere MLX works — LM Studio, mlx-vlm, and friends — at stock speed, with the vision tower fully functional and coherent thinking traces. If you just want to try it: download the repo below, point LM Studio at it, done. HuggingFace Links WinterMix58 — 82 GiB, ~6.0 bpw: the best-measuring MLX quant of this model I'm aware of at any size, including against 94–95 GiB 6-bit oMLX builds (narrowly at 2K, more clearly at 16K). WinterMix48 — 68 GiB, ~5.0 bpw: leaves ~35–40 GB free on a 128 GB Mac = 5–8 parallel 100K-token agent sessions resident at once (GDN architecture keeps a 100K session's cache at ~5–10 GB). Beats its direct size-peer (oQ4, 67 GiB) by ~1.4–1.5% at both context lengths. Numbers One scoring rule for every row (NLL over the second half of each window, token-aligned across engines — llama.cpp's native rule, so these are comparable to Unsloth's), paired per-token where both models run under MLX. Reference rows were measured on my own harness: same tokens, same machine. oMLX quants are included because oMLX is currently the popular option for MLX. All rows are Qwen3.5-122B-A10B in various quantization mixes. model GiB short-2K ppl long-16K ppl Unsloth UD-Q5_K_XL GGUF (llama.cpp) 85.6 4.2343 4.3845 6-bit-expert RTN transfer (MLX) 95 4.2504 4.4424 oQ6 (oMLX) 94 4.2538 4.4172 WinterMix58 82 4.2481 4.4149 oQ5 (oMLX) 80 4.2904 4.4493 WinterMix48 68 4.3276 4.5038 oQ4 (oMLX) 67 4.3933 4.5679 Being upfront about the ceiling: the imatrix-rounded source GGUF is still slightly ahead (+0.3–0.7% rule-matched). Matching imatrix-style weighted rounding in MLX would need custom inference kernels, and "loads in everything at stock speed" was a hard constraint I wasn't willing to break. Within the native format, this appears to be about the limit. The part I think is actually interesting Halfway through this project I found that perplexity is blind to real behavioral differences between quants. Two builds with statistically identical NLL differed 2.5× in how often they self-interrupt ("wait, let me re-check...") during 50K-token reasoning traces. Then the reverse bit me: my best-NLL build had an elevated self-interruption count — and actually reading the traces showed it wasn't confusion at all, but disciplined audit passes that twice caught a base-model reasoning bug before the final answer. So the release models were selected on three instruments: paired NLL, blind-scored state-tracking benchmarks at depth, and directly reading the reasoning traces. Both releases deliver perfect scores on a 30-step adversarial state-tracking task on every seed — and the 68 GiB build's traces show it catching its own 4-bit arithmetic slips before they reach the output. If you evaluate quants, I'd honestly recommend reading traces over counting anything. What's under the hood (briefly) Sensitivity-informed mixed-precision allocation (routing-critical tensors pinned at BF16 — MoE routers do not like being quantized), GPTQ-family error-compensated rounding reimplemented natively for the MLX affine format and executed layer-wise (whole-model GPTQ OOMs a 122B on 128 GB; streaming it peaks around 28 GB), and a diverse long-context calibration mixture engineered so every expert in every layer actually gets calibrated — including multilingual content, because it turns out an English-only calibration set silently starves the language-specialist experts. Validated across 18 measured variants with paired controls and held-out out-of-domain checks (no calibration binding: code/math within ±0.1% of RTN). I'm not releasing the pipeline code for now — the models are open weights (Apache 2.0), the method writeup stays private. The M5 Max kernel-panicked ten times during development before I got the workload tamed, if that helps set the vibe. Requests I'm planning to take requests for MLX quantizations of other models — drop them in the comments or in the HF Community tabs. Practical constraints: it has to fit the pipeline on a 128 GB Mac (up to ~120B+ MoE is proven), and dense models calibrate differently than MoE, so results may vary until I've tuned per-architecture. Happy to answer questions about the eval methodology, the behavioral testing, Apple Silicon quirks (ask me about watchdog panics), or Mac long-context agent setups. submitted by /u/WinterCharm [link] [comments]

[[paper-statistically-lossless-quantization-of-large-language-|[Paper] Statistically-Lossless Quantization of Large Language Models]]
Now Suddenly too many choices for DGX Spark with Qwen 3.5 122B . What would be the next upgrade?
DeepSeek V4 Flash, up to 32 tok/s on AMD Ryzen AI MAX+ 395

Source: r/LocalLLaMA | 2026-08-02

Related