ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level

arXiv:2607.13511v1 Announce Type: cross Abstract: We introduce ExTernD (Expanded-rank Ternary Decomposition), a post-training factorization of each LLM weight matrix A in R^{m imes n} into A approx B diag(D) C with ternary factors B in {-1,0,+1}^{m imes k}, C in {-1,0,+1}^{k imes n} and a real scale vector D in R^k. The inner rank k = mu min(m,n) is deliberately expanded beyond full rank (mu > 1), so that components past full rank correct the quantization error of earlier ones. We prove the residual decreases monotonically in k and can be driven below any arepsilon > 0: ExTernD approaches bf16 accuracy arbitrarily closely, which no ternary scheme with a fixed plane count can do. Memory and compute scale continuously with mu, and factor sparsity continuously with a threshold au, so an accuracy target is hit exactly rather than rounded to the next bit-width. ExTernD matches Q4_K's per-matrix accuracy at 5.2-5.5 effective bpw (5.1-5.5 with importance weighting) on Gemma-4-E2B and Qwen3.5-4B, and a full Qwen3.5-4B conversion at mu = 3 reaches 10.10 wikitext-2 perplexity against 9.78 for bf16 (+3.2%), placing it near the Q4_K/Q5_K accuracy band at ~5.7 effective bpw.

Source: arXiv cs.AI | 2026-07-16