Model Releases
Based on advice from LocalLLama implemented a n+2 lookahead prediction for ssd streaming Qwen Flash Next - and it worked!
My project is optimising qwen flash next on Mac Mini m5 64gb through SSD streaming. I recently asked LocalLLama for guidance on how to optimise token output. Today a popular method to do this is a loo
My project is optimising qwen flash next on Mac Mini m5 64gb through SSD streaming. I recently asked LocalLLama for guidance on how to optimise token output. Today a popular method to do this is a lookahead guesser which fetches experts one layer ahead of the current layer. One popular suggestion was building a prediction model to start loading experts two layers ahead. This netted me +8% on chat and +4.2% on code. The way the various optimisations stack on top of each other this may actually net alot more on a base vanilla setup. https://github.com/skeggsguy/Flash-next-ssd Pretty cool! Thanks reddit LocalLLama! I’ll go find the og post to link now. Edit: Kudos to sdroege_ for og suggestion! submitted by /u/turtleninja99 [link] [comments]
Related
- I got Qwen Flash Next Q4 running on a Mac Mini m5 64gb with ssd streaming
- 10%+ performance improvement on MoE ssd-streaming with expert-lookahead
- MoE SSD streaming on a 64 GB Mac mini: GPU still waits 27% of decode on experts. Ideas?
Source: r/LocalLLaMA | 2026-10-13