Model Releases
Need help fixing Gemma 4 26B A4B toolcalls in Pi
Hello! So, yesterday i tried out Gemma 4 26B A4B QAT Q4_K_XL from here with Q4_0 MTP model in Pi agent. Suddenly, it became the first model to actually pass my simple test with HTTP server in C++, whi
Hello! So, yesterday i tried out Gemma 4 26B A4B QAT Q4_K_XL from here with Q4_0 MTP model in Pi agent. Suddenly, it became the first model to actually pass my simple test with HTTP server in C++, while being small enough to fit 128K ctx at FP16. It easily oneshotted the task, tested, and it worked, all of that in just 5 minutes! It was hella impressive, and i decided to push further. It started experiencing many issues with toolcalling. It fails to submit the plan to Pi. It fails to edit or write the file because it mixes up the arguments or forgets some. It became really annoying when it got to ~30K of context, and i decided to stop. What can I do to improve toolcalling success rate? CLI flags for llama-server: build/bin/llama-server -m ~/d/models/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 131072 -ngl all --cpu-moe --no-host -np 1 --cache-ram 0 -fit off -lv 4 --no-webui --no-mmproj --spec-type draft-mtp -md ~/d/models/mtp-gemma-4-26B-A4B-it-Q4_0.gguf --spec-draft-n-max 3 --n-gpu-layers-draft 99 --load-mode none -ub 4096 --temp 1.0 --top-k 64 --top-p 0.95 --min-p 0.05 --repeat-penalty 1.1 --repeat-last-n 256 --reasoning-budget 12288 It also repeated a lot, so i had to apply recommended settings. I check chat template, and looks like Unsloth applied fixed template that was released on July. submitted by /u/HyperWinX [link] [comments]
Related
- Gemma 4 QAT could be improved further by Google aligning the QAT model to modern q4_k instead of q4_0
- Cactus Hybrid: We taught Gemma 4 to know when it's wrong
- Scotoma-2: Gemma4, but with less annoying slop and better writing.
Source: r/LocalLLaMA | 2026-09-20