Model Releases
ThinkingCap-Qwen3.6-27B warrants a look
It has only been two days since I move 100% from Qwen3.5-27B F16 to ThinkingCap-Qwen3.6-27B F16. Where I was getting tps in 30-40 range (depending on the size of the context), I am definitely getting
It has only been two days since I move 100% from Qwen3.5-27B F16 to ThinkingCap-Qwen3.6-27B F16. Where I was getting tps in 30-40 range (depending on the size of the context), I am definitely getting 35-45 range. Not much of a bump you may say but I have not noticed any loss in quality. They claimed to have reduced token usage. Maybe that is what is translating into the higher tps. Key is that they did not mess up the brains. The chat template is froggeric. I am sold. This is what I will use till Qwen drops another one. spec-draft-n-max 4 works best. I have tried from 1-6. Here is my llama script CUDA_VISIBLE_DEVICES=3,2,1,0 ~/llama.cpp/build/bin/llama-server -m ~/models/ThinkingCap-Qwen3.6-27B/ThinkingCap-Qwen3.6-27B-f16.gguf --port 8000 -c 262144 -b 4096 -ub 512 -np 2 -ctk f16 -ctv f16 -ctkd f16 -ctvd f16 -fa on -ts 1,1,1,1 --spec-type draft-mtp --spec-draft-n-max 4 --reasoning on --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 --repeat-penalty 1.1 --presence-penalty 0.1 --alias Unsloth/ThinkingCap-Qwen3.6-27B-f16 --host 0.0.0.0 --no-ui --jinja --chat-template-file ~/models/Qwen3.6/chat_template.jinja Would love inputs on what I could change to get "mo" tps. submitted by /u/TinyFrodo [link] [comments]
Source: r/LocalLLaMA | 2026-07-28