I trained a 1B-parameter LLM from scratch on 20B tokens for about $200
A few months ago, I had the idea of making a LLM from scratch as a personal project (for learning and partly for improving my resume). Since I learned a lot from other posts on here over the past year