GLM 5.2 and ik_llama.ccp

Running GLM-5.2 (the new glm-dsa arch), Unsloth UD-Q4_K_XL, on a 4-socket Xeon E7-8880 v4 box with 1TB RAM and a single RTX 3060 12GB. ik_llama.cpp, experts on CPU (--cpu-moe), 24 attention layers on the GPU. Works great at 8k context — rock solid, ~3.7 tok/s gen. Problem: the second I raise context (anywhere past ~32–64k), generation crashes on the very first token. Fatal error in llama-sampling.cpp, and the dumped probabilities.txt shows every logit is nan (max = nan, sump = nan). 8k always works, long context always dies. Stuff I've already tried, none of it helped: -dsa on and off -fa off (and on) shrinking -ub all the way down to 32/64 -nkvo (KV on CPU) both f16 and q8 KV cache pulled the latest commits with the GLM-DSA indexer fixes (#2075, #2090, #2093) Same all-NaN crash every time. Feels like an f16 overflow somewhere in the DSA/indexer path on the GPU, but I can't flag my way around it. Model's trained for 1M context so I'd really like to actually use it. Is this a known glm-dsa long-context bug? Any workaround to keep it on the GPU, or is CPU-only genuinely the only path right now? submitted by /u/naunen [link] [comments]

Related

• Running Qwen 3.6 35b MoE With Zoo Code On M1 Max is Amazing! Fully local, battery-powered coding powerhouse!
• Running Qwen 3.6 35B MoE (Q4_K_M) on a Zeus (Xiaomi 12 Pro, 12GB RAM)
• NCCL-Free Tensor Parallelism on Dual Blackwell PCIe llama.cpp b9095 released!
• Pipeline parallelism in llama.cpp may be wasting your VRAM

Source: r/LocalLLaMA | 2026-07-26