100B Models on Cheap Hardware: how realistic and limitations
DGX agentThere is a lot of buzz around running 100B parameter models on cheap local hardware using ternary (1.58-bit) quantization like Microsoft's BitNet architecture. The theoretical hardware shortcuts are i