Lecture: Building a GPU Compiler From Scratch. Lowering PyTorch Graphs to CUDA via Multi-Level IRs (GPU_MODE video + article)

https://preview.redd.it/3ekngxk224vh1.png?width=1080&format=png&auto=webp&s=9bfe5c92033e17d5b2fb4529f1320c99455a138b I built a GPU compiler and wrote an article about it, focusing on concepts shared by modern compilers like TVM, Modular, and PyTorch Inductor. The compiler works with vLLM for full end-to-end model serving, so it may be useful if you're interested in building LLM serving engines. The article breaks CUDA kernel generation into a sequence of passes and shows how a common operation (RMSNorm, Matmul, Attention) is progressively lowered to generate a CUDA kernel. Each pass introduces its own IR that is optimized for specific tasks like kernel fusion or schedule enumeration. https://preview.redd.it/tm16quo324vh1.png?width=1080&format=png&auto=webp&s=4435d5b7f7aacd90de2841731597ae77f2ecbca7 Each example is reproducible, so you can write PyTorch code and generate code at a specific IR level, which makes the flow easier to understand. % emmy compile -c 'nn.Linear(3, 2, bias=False)(torch.randn(4, 3))' --ir loop for a0 in 0..4 for a1 in 0..2 for a2 in 0..3 in0 = load input[a0, a2] in1 = load linear_wt[a2, a1] v0 = multiply(in0, in1) acc0 <- add(acc0, v0) linear[a0, a1] = acc0 On the benchmarked workloads, many outperform Torch Inductor and cuBLAS. Generated kernels use modern features like tensor cores and TMA. https://preview.redd.it/jmbmyc1624vh1.png?width=1080&format=png&auto=webp&s=6ed1559514475626d9265725a01b0ec47c4d8d9a The field is very dynamic, with many competing approaches, so it's hard to pin down the "core concepts"; this is my take on them. YouTube: https://youtu.be/KK6TWf4Pp6A?si=0hCuOMEan4mauJjN Article: https://riftstack.ai/research/inside-a-gpu-compiler-from-pytorch-to-cuda GitHub: https://github.com/cloudrift-ai/emmy submitted by /u/NoVibeCoding [link] [comments]

Related

• microsoft doubles down on local ai with nvidia, but with a cost
• The page opened in 1.1s, but my local agent spent 53.5s reading it
• First results from H3 Acceleration Arena

Source: r/LocalLLaMA | 2026-10-12