Hardware
Nice showcase that interesting LLM work can be done on single GPU!
Nice showcase that interesting LLM work can be done on single GPU! I extended the GPT-2-style code from @rasbt's 'Build a Large Language Model (from Scratch)' so that it was a 6-expert (2 active) mixt
Nice showcase that interesting LLM work can be done on single GPU! I extended the GPT-2-style code from @rasbt's "Build a Large Language Model (from Scratch)" so that it was a 6-expert (2 active) mixture-of-experts, and trained it from scratch over 8 days. It worked well! Full writeup with maths and code at https://www.gilesthomas.com/2026/09/gp…
Related
Source: Sebastian Raschka (X) | 2026-09-10