Sherry's 3:4 ternary format (1.375 bits per weight) running on WebGPU: a 1.6 MB model that plays Connect Four as well as its 7.8 MB int8 version

Not an LLM, but the ternary findings should carry over, and we hadn't seen Sherry-style 3:4 weights run in a browser before. Disclosure: this is our work at Precisit, everything is MIT. What it is A 7.4M-parameter one-pass scorer (the jevlike family): the board goes in, one score per legal column comes out. No search. Weights in T34, Sherry's 3:4 format: in every four weights one is zero and three are ±1, so four weights fit in 5 bits. One fp16 scale per 128 weights gives 1.375 bits per weight. The embedding is int8; norms and biases are fp16. It runs in the browser on a small WebGPU runtime: 1.1 ms per move (idle M5 Pro, Chrome). Model File size vs depth-4 bot vs depth-6 bot dense (fp32) 29.7 MB 0.92 0.89 T34, trained ternary 1.59 MB 0.93 0.91 T34, fine-tuned from dense 1.59 MB 0.89 0.9 T34, converted after training 1.59 MB 0.13 0.11 Base243 (TQ1_0 style), trained 1.93 MB 0.89 0.88 200 games each, both sides play a random move 5% of the time, a win counts 1 and a draw ½. What we learned Converting the finished model to 3:4 collapsed it (0.13 against the depth-4 bot). Training with the format in the forward pass fixed it completely, whether from scratch or fine-tuning. Attention's q/k/v matrices are the sensitive ones. Group size (64/128/256) barely mattered. Seeds matter: two runs of the same T34 recipe scored 0.945 and 0.882. Play it: https://precisit.github.io/onepass-web/demo/c4-size/ Code, models, every result: https://github.com/precisit/onepass-webgpu-ternary The write-up: https://precisit.com/en/blog/onepass-c4-size/ Has anyone gotten post-training 3:4 conversion to work on models, or does it need training? submitted by /u/Brilliant-Hall1387 [link] [comments]

Source: r/LocalLLaMA | 2026-09-29

Related