Interesting surprise drop from Thinky! The Inkling model looks pretty solid on benchmarks, and it has some little surprises in its architect…

Interesting surprise drop from Thinky! The Inkling model looks pretty solid on benchmarks, and it has some little surprises in its architecture: - Small conv layers in several places - An RMSNorm for the embeddings (before the block RMSNorm) - Rel. position bias instead of RoPE first open weight thinking machine model!! 975B total, 41B active trained on 45T tokens, 1M context, multimodal in sliding window with a 5:1 ratio and 512 size, deepseek aux-free load balancing and 2 shared experts (usually people only use 1), actually curious why the model is le…

Source: Sebastian Raschka (X) | 2026-07-15