Qwen3.8-27B on M1 Max 32GB: MLX 15.8 tok/s vs llama.cpp 9.7 tok/s - but llama.cpp prefill is faster

I’ve been setting up an M1 Max Mac Studio (24-core GPU, 32GB unified memory) as a local LLM server and wanted to compare MLX vs llama.cpp on Qwen3.8-27B. I tried to keep the model footprint and benchmark workload reasonably close. MLX mlx-community/Qwen3.8-27B-4bit ~16.1GB 512 prompt tokens / 700 generation tokens 3 runs Prompt: 81.76 tok/s Generation: 15.81 tok/s Peak memory: 16.39GB llama.cpp unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M 15.32 GiB / 27.32B params Full Metal offload Flash Attention enabled 512 prompt tokens / 700 generation tokens 3 runs Prompt: 99.61 ± 0.44 tok/s Generation: 9.69 ± 0.34 tok/s So on this machine: llama.cpp is ~22% faster for prompt processing, while MLX is ~63% faster for autoregressive generation. submitted by /u/Substantial_Camp1317 [link] [comments]

Source: r/ollama | 2026-09-07