Qwen3.8-27B on a 24GB M4 Pro Mac mini: benchmarks and the three settings that stop it drowning

When Qwen3.8-27B dropped on Thursday the obvious question came up for us : does a 27B dense model actually fit on the 24GB Mac Mini machines? Ran it properly over the weekend on an M4 Pro (24GB unified, macOS 26.6.1, llama.cpp b10488, bartowski GGUFs). Numbers and gotchas below. Speed (llama-bench, 3 runs each, flash attention on, full GPU offload): Quant File size pp2048 tg128 Q4_K_M 17.77GB 96.8 tok/s 11.4 tok/s IQ4_XS 15.57GB 95.4 tok/s 10.9 tok/s Basically identical speed, so take IQ4_XS: the 2GB you save goes straight into context headroom. Q5_K_M and up, forget it on 24GB. The three settings that matter: Raise the GPU wired limit. macOS won't wire 17.8GB to the GPU by default on a 24GB machine. sudo sysctl iogpu.wired_limit_mb=20480 fixes it (resets on reboot). After this, Metal reports a 21.5GB working set and the whole model loads clean. Quantise the KV cache for context. With IQ4_XS + -fa 1 -ctk q8_0 -ctv q8_0, a full 32k context loads and answers fine: ~16.6GB resident, 20% of system memory still free. Turn thinking off for interactive use. This is a reasoning model and at 11 tok/s that hurts: our first coding prompt produced 6,500 characters of chain-of-thought and hit a 1,600-token cap without ever starting the answer, 142 seconds of deliberating. With "chat_template_kwargs": {"enable_thinking": false} in the llama-server request, the same prompt returned a complete working Python tool in 28 seconds. Keep thinking on for batch/overnight jobs where the quality gain is worth the wait. One more thing: llama-cli with a raw -p prompt ran away on us and dumped gigabytes of output. Use llama-server, it handles the chat template properly and you get the web UI for free. Verdict: genuinely usable at 4-bit, ~11 tok/s is slow but okay for off line tasks and just about bearable reading speed. Q4 output quality is strong. But it's the ceiling for 24GB: no room for Q8 (28.6GB), no room for the vision encoder next to a big context, no room for the rest of your stack. The full writeup with methodology is here: https://halfpennymac.com/qwen38-27b-m4-mac-mini-benchmark Happy to answer questions or rerun anything with different settings, the test machine is sitting next to me. submitted by /u/halfpennymac [link] [comments]

Related

• Qwen3.6 vs gpt-oss:120b on Apple Silicon — three Qwen variants benchmarked, plus what works and where it does not
• Ollama + Qwen
• Compared qwen3.6, qwen3-coder, and deepseek-coder on three coding benchmarks. All running locally on Ollama
• Hey all im on a m5 pro macbook pro and i am just trying to runollama run qwen2.5-coder:14b - now I could be doing something wrong but keep facing error 500 or eof so i tried a llama 3 1b model and i still get error 500. i have tried restarting and deleting ollama and so much more but nothing....

Source: r/ollama | 2026-08-18