Search: “sebastian-raschka--x”

Sebastian Raschka shared an HTML-formatted table comparing active parameter ratios across different large language model architectures for improved readability and to avoid text truncation. The resour

A little talk on what we can learn from implementing LLM architectures from scratch in Python and PyTorch. And how I approach new open-weight models, compare them against reference implementations etc

Interesting paper. What I like about this is that it is a relatively low-commitment attention modification. I.e., one can use it during most of training, switch back to vanilla attention near the end,

Back from a little family break! Lots has happened, and I’m planning to do a deeper dive into the most interesting architectural components (soon). Btw, are there any major architectures I missed belo

Sebastian Raschka maintains a comprehensive gallery of LLM architecture diagrams and visualizations on his website, with higher-resolution versions available at sebastianraschka.com/llm-architecture-g

This post documents a second wave of large language model and AI architecture releases from April, featuring updates from multiple organizations including Ant Ling 2.6 1T, Minimax M2.7, Xiaomi MiMo V2

April saw significant activity in large language model releases, with five major models introduced including Gemma 4, GLM-5.1, Qwen 3.6, Kimi K2.6, and DeepSeek V4. These releases have been added to a

Sebastian Raschka has updated his LLM architecture gallery with higher resolution figures and improved summaries, including coverage of the DeepSeek V4 model architecture. This resource provides visua

Had a great time at PyCon & PyData DE. Highly recommend it. Great open-source, community-focused conference with lots of builders in the Python AI, LLM and agent space. Taking a short family break, my

I’ve uploaded a new paper on arXiv (co-authored by @rasbt): MiCA Learns More Knowledge Than LoRA and Full Fine-Tuning In Parameter-Efficient Fine-Tuning, a key question may not just be how low-rank th

Sebastian Raschka (rasbt) used Randal Olson's Tufte Test tool, developed by Goodeye Labs, to improve the visual quality of a machine learning benchmark plot. The Tufte Test encodes seven of Tufte'...

Strong release! GLM-5.1 is a DeepSeek-V3.2-like architecture (including MLA and DeepSeek Sparse Attention) but with more layers. And the benchmarks look better throughout! Looks like THE flagship open