Search: “sebastian-raschka”

A lot of hype around OpenAI's Astra model here on my timeline today. Apparently, this goes back to a new article from The Information, which said Astra is a 'recurrent depth or looped transformer'. It

Sebastian Raschka’s September 2026 article introduces **GPT‑6 *Astra***, a next‑generation language model that expands upon the Transformer backbone with a novel *looped* architecture—allowing multipl

GPT‑6 Astra is OpenAI’s latest flagship model, outperforming GPT‑5.6 across virtually all common benchmarks—most noticeably in math, coding, and especially 3‑D rendering/animation tasks. The architect

A little Ember-1 tl;dr. Seems like a great model! (Was recently asked on a podcast, given $ xx million, what's the best way to develop a frontier LLM today? My recommendation was: start with an existi

During a Swyx podcast interview in October 2026, an OpenAI engineer revealed that the company’s Decision API is built on the GPT‑6 Luna model, augmented with a dedicated decision head. This configurat

Sebastian Raschka has updated his LLM architecture gallery with higher resolution figures and improved summaries, including coverage of the DeepSeek V4 model architecture. This resource provides visua

Just saw that the LLMs-from-scratch repository passed 100,000 stars on GitHub! This is super cool and motivating. I am really happy to see that this open-source repo has helped so many people. Thanks

Whoa, Meta released a new open-weight LLM yesterday, something that hasn't happened since the good old Llama days. Their Meta Muse Glimmer model is a 30B multimodal reasoning model with a Gemma-like a

Yes, open-source / open-weight models are important for a healthy AI ecosystem. That's how we can verify things, check claims, and keep up outside the closed labs. Plus, it gives us the freedom to run

Also kudos to fireworks AI for admitting that they built on top of Kimi (I remember the awkward Mistral 3 release) https://x.com/rasbt/status/1999557830131494977?s=20 Hold on a sec, Mistral 3 Large us

Interesting surprise drop from Thinky! The Inkling model looks pretty solid on benchmarks, and it has some little surprises in its architecture: - Small conv layers in several places - An RMSNorm for

Yes, open-source / open-weight models are important for a healthy AI ecosystem. That's how we can verify things, check claims, and keep up outside the closed labs. Plus, it gives us the freedom to run

The Kimi K3 architecture figure for yesterday's big open-weight model release, along with some observations and thoughts. 1. Yes, it looks relatively complicated, but it's essentially a scaled-up prod

Just saw that the LLMs-from-scratch repository passed 100,000 stars on GitHub! This is super cool and motivating. I am really happy to see that this open-source repo has helped so many people. Thanks

Now we know: The popular Ox Alpha LLM was GLM-5.3-Flash... Compared to GLM-5.2, this new GLM-5.3-Flash model uses: - a Kimi Linear-style 3:1 (super*) hybrid attention pattern with 34 Kimi Delta Attent

Big overhaul on DeepSeek V4.1 using an encoder-decoder setup. Tbh they should have called it DeepSeek V5! Super cool and refreshing, though! 🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more ef

Also kudos to fireworks AI for admitting that they built on top of Kimi (I remember the awkward Mistral 3 release) https://x.com/rasbt/status/1999557830131494977?s=20 Hold on a sec, Mistral 3 Large us

Reasoning from scratch, round number 6! An introduction (and implementation) of Reinforcement Learning with Verifiable Rewards (RLVR) and Group Relative Policy Optimization (GRPO). 00:00 Introduction

And another open-weight release. Nemotron 3 Ultra has an ultra impressive capability:efficiency ratio! Design-wise, it carries forward the Mamba-2-attention hybrid stack and LatentMoE introduced in th

The Kimi K3 architecture figure for yesterday's big open-weight model release, along with some observations and thoughts. 1. Yes, it looks relatively complicated, but it's essentially a scaled-up prod

Whoa, Meta released a new open-weight LLM yesterday, something that hasn't happened since the good old Llama days. Their Meta Muse Glimmer model is a 30B multimodal reasoning model with a Gemma-like a