Local Ai
When will we have another breakthrough for faster local dense models?
Most of the speedup are touching along the likes like cutting the active params reletive to total params(MOE) but it does take a hit in stability & reliability compared to a dense-27b can be much more
Most of the speedup are touching along the likes like cutting the active params reletive to total params(MOE) but it does take a hit in stability & reliability compared to a dense-27b can be much more stable at low total params. We hade the innovation of MTP which is great and lots of other things like strata coming along but ever since MTP dense models havent gotten any architectural attention to speedups other than reduced reasoning tokens from community finetunes. with qwen4-27b coming up do we have any chance of any speedup breakthroughs? submitted by /u/Express_Quail_1493 [link] [comments]
Related
- 40% speedup of MoE training with faster megakernel, by cursor, of all people (for B200s)
- The idea: on a CPU the decode speed depends on the active params per token, not the total. My objective is trying to run a 10B at 100tok/s on a mid level PC (No GPU).
- Spent two weeks on a kernel that benchmarked 29x faster. End to end it's maybe 6-10%, and it's not even wired in yet.
- For MoE models the arithmetic splits in two: capacity follows total params, speed follows active
Source: r/LocalLLaMA | 2026-10-22