Tools
eugene is doing a great job covering it on the @latentspacepod paper club (will post on the youtube soon) https://x.com/picocreator/status/2…
eugene is doing a great job covering it on the @latentspacepod paper club (will post on the youtube soon) https://x.com/picocreator/status/2047626012419313889?s=20 🗒️ Other things - FP4 MoE training -
eugene is doing a great job covering it on the @latentspacepod paper club (will post on the youtube soon) https://x.com/picocreator/status/2047626012419313889?s=20 🗒️ Other things - FP4 MoE training - Muon Optimizer 🫶 - No linear attention 💔😭 - Partial Rotary Embedding - attention sinks! with CSA & HSA - mixed KV precision, FP8 native attention! (kind of) - same MTP - layers are alternated : CSA then HCA - last layer is full MLA
Source: Swyx (X) | 2026-04-29