Model Releases
Ternary Bonsai 2 Family: 2B, 4B, & 9B based on Qwen 3.5
Going to try these out to see if they have the same looping behaviors that other Bonsai models have had. Still requires their fork of llama.cpp (branch prism) so dont expect it to work with vanilla ll
Going to try these out to see if they have the same looping behaviors that other Bonsai models have had. Still requires their fork of llama.cpp (branch prism) so dont expect it to work with vanilla llama.cpp. Edit: You can use with regular llama.cpp! Will see if there is some way to support this for AnythingLLM Mobile via LLama.rn fork since if anything this will hopefully beat out regular Qwen 3.5 1.7B or 0.8B. Can test on iOS today with LocallyAI on latest version. Feel like the way these models are made (QAT?) kind of lend to a high benchmarked model that is quite different when deployed. Past experiences are certainly better with memory for the parameter size but let's see how these go. Says dFlash and MTP supported as well + MLX support to juice some more TPS submitted by /u/tcarambat [link] [comments]
Related
- 1-bit / 2-bit / Ternary / Bitnet Models - Updates & Tracking
- Ternary Bonsai 2 (2B, 4B, and 9B) just released on Hugging Face. The 2B can run entirely locally in the browser at up to 500 tok/s on an M4 Max with custom WebGPU kernels.
- Prism-ML Bonsai 2 Joins Our Qwen3.8 Quantization Comparison
Source: r/LocalLLaMA | 2026-10-15