Model Releases
The CEA architecture is a bigger deal than I initially thought
I initially saw CED as just an efficiency improvement, but the more I read about it, the more it feels like an inference architecture leap. The encoder/decoder split has some pretty interesting implic
I initially saw CED as just an efficiency improvement, but the more I read about it, the more it feels like an inference architecture leap. The encoder/decoder split has some pretty interesting implications for GPU pooling. Instead of treating every GPU the same, you could have prefill-specialized GPUs for the encoder and decode-specialized GPUs for the decoder, each optimized for a different part of inference. Or just using more modern GPUs for the prefill phase and old HBM cards for decode in a heterogenous setup. 4.1 Flash obviously won't fit on my 4× MI50 + 2× V620 setup, but if Qwen adopts this in a new Flash model I'd be so pumped. submitted by /u/metmelo [link] [comments]
Related
- GLM-5.2 local inference: ubatch size made a much bigger difference than I expected
- Question: Why is prefill unbelievably faster in vLLM than other inference engines?
- 1M context with 17 GB model in 24 GB VRAM: 'for the first time I was able to load a context of almost 1M tokens and extract 7 needles from various parts of the text'
Source: r/LocalLLaMA | 2026-09-10