Model Releases
boro64/Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF · Hugging Face
Ternary Nemotron 30B-A3B — 368K Context on 12 GB GPU (GGUF) A ternary (1.58-bit) 30B-A3B MoE model that runs on a 12 GB GPU with 368K context, 172 tok/s decode and 7000+ tok/s prefill — in both Stock
Ternary Nemotron 30B-A3B — 368K Context on 12 GB GPU (GGUF) A ternary (1.58-bit) 30B-A3B MoE model that runs on a 12 GB GPU with 368K context, 172 tok/s decode and 7000+ tok/s prefill — in both Stock llama.cpp and the PrismML fork. One file, both engines. Engines ✅ Stock llama.cpp (validated on b11555+, b11796) · ✅ PrismML fork / Bonsai (validated on b10793) EDIT: This is the 2nd Ternary MOE after maple-preview which works with stock llama.cpp. Somebody please share CPU-only numbers. Also share numbers of tiny VRAMs(~10GB) submitted by /u/pmttyji [link] [comments]
Related
- Nemotron-3.5-Lightning at 11.77 GiB, a 16 GB option for a model that didn't have one
- nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB · Hugging Face
- nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB · Hugging Face
Source: r/LocalLLaMA | 2026-10-26