Model Releases
M1 Max 32GB, trying to run Qwen 3.8 27B at decent speeds and context
I would like to stay at q4, and serve the model to various coding harnesses. None of the models or tools I have tried will run without running out of memory. I am hoping for some advice. oMLX? MTPLX?
I would like to stay at q4, and serve the model to various coding harnesses. None of the models or tools I have tried will run without running out of memory. I am hoping for some advice. oMLX? MTPLX? llama.cpp? Has anyone gotten a good Qwen 3.8 27B running ok on an M1 Max 32GB MacBook Pro? Edit: Decent speeds means it's not all the way down at 1 t/s. And if anyone has other suggestions to get close to the quality but with another model, I'd love suggestions. submitted by /u/greggh [link] [comments]
Related
- Running Qwen 3.6 35b MoE With Zoo Code On M1 Max is Amazing! Fully local, battery-powered coding powerhouse!
- Qwen-Next seems worse to me then 3.8 27b for coding, but I feel like I must be missing something?
- Need to decide: DGX spark vs framework desktop vs Mac mini/studio
Source: r/LocalLLaMA | 2026-09-19