Model Releases
you can now use MTP in GLM-Air
If anyone still remembers GLM-4.5-Air from last year, you can now get a nice speedup by enabling MTP in llama.cpp. It is a 106B MoE with only 12B active parameters, which makes it interesting for mach
If anyone still remembers GLM-4.5-Air from last year, you can now get a nice speedup by enabling MTP in llama.cpp. It is a 106B MoE with only 12B active parameters, which makes it interesting for machines with lots of memory but limited compute, such as Strix Halo or DGX Spark. I use it on 3090s. It's still great for creative writing, especially since we never got Gemma 4 124B MoE. There are multiple creative-writing / RP finetunes available on Hugging Face: https://huggingface.co/models?other=base_model:finetune:zai-org%2FGLM-4.5-Air&sort=likes (some even from this year). I also recommend Intellect 3.x by PrimeIntellect If your GGUF does not include the MTP block, you can download a small file from here: https://huggingface.co/jacek2024/GLM-4.5-Air-MTP-GGUF Thanks a lot to devMiikaK and HeadCutter for testing the PR while it was in progress. PS. It also works for the full GLM-4.5, but I doubt anyone still uses it ;) submitted by /u/jacek2023 [link] [comments]
Related
- 80 tok/sec and 128K context on 12GB VRAM with Qwen3.6 35B A3B and llama.cpp MTP
- SyzygyResearch/Mach-1-Additive-35B-GGUF · Hugging Face
- Inkling-Small by thinkingmachines
- A llama.cpp PR caches “hot” MoE experts on the GPU — 33 → 56 tok/s reported with 8GB VRAM
Source: r/LocalLLaMA | 2026-08-23