Research
[P] Building a LLM from scratch with Mary Shelley's 'Frankenstein' (on Kaggle)
A beginner-friendly tutorial demonstrating how to build a ~3.2M parameter LLM from scratch using Mary Shelley's *Frankenstein* as the sole training corpus, designed to run on Kaggle's free GPU in u...
A beginner-friendly tutorial demonstrating how to build a ~3.2M parameter LLM from scratch using Mary Shelley's Frankenstein as the sole training corpus, designed to run on Kaggle's free GPU in under 20 minutes. The resulting model is a raw, base-level GPT-style transformer trained via character-level tokenization with a 256-token context window — it performs next-token prediction and prompt completion without any fine-tuning or RLHF. The project serves as an accessible, end-to-end educational resource for understanding core LLM concepts such as tokenization, attention mechanisms, and hyperparameter configuration, with no prior programming experience required.
Related
- Started a video series on building an orchestration layer for LLM post-training [P]
- AI Systems Performance Engineering by Chris Fregly - is it worth it? [D]
- [[d-60-matmul-performance-bug-in-cublas-on-rtx-5090-d|[D] 60% MatMul Performance Bug in cuBLAS on RTX 5090 [D]]]
- [[p-pca-before-truncation-makes-non-matryoshka-embeddings-comp|[P] PCA before truncation makes non-Matryoshka embeddings compressible: results on BGE-M3 [P]]]
Source: research