Local Ai
What quant do you actually use day to day? Anyone happy with 2-bit or 3-bit?
Curious what everyone runs in practice: - What quant do you use most, and for what (chat, coding, agents)? - 2-bit: do you use it? Is it actually usable or too broken? - 3-bit: good enough for coding
Curious what everyone runs in practice: - What quant do you use most, and for what (chat, coding, agents)? - 2-bit: do you use it? Is it actually usable or too broken? - 3-bit: good enough for coding and more complex stuff, or do you go back to 4-bit? - If you look at KLD: what number do you call "basically lossless"? Is 0.1 already good, or do you want something like 0.01-0.03? Real experience over benchmarks. Thanks! (English isn't my first language, so I used an LLM to help translate this post.) submitted by /u/Queasy_Back2588 [link] [comments]
Related
- What is your best coding model on a DGX Spark?
- Understand Kimi K3 from first principles: a recommended order for anyone trying to understand this beast
- My Reading Library: Evaluating LLMs on Android Tasks
- Has anyone experimented with stabilizing low quant models with lower temp and top p?
Source: r/LocalLLaMA | 2026-10-18