ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights

arXiv:2609.38521v1 Announce Type: cross Abstract: We introduce ShamAN-Q, a sub-1-bit post-training quantization method that extends NanoQuant by replacing each its diagonal reconstruction geometry with a tractable dense curvature metric, using a general paradigm popularized by the Shampoo optimizer. For each linear weight, ShamAN-Q fits a Kronecker product to the empirical Fisher information matrix of a small calibration set by Kullback--Leibler minimization, forming a Mahalanobis reconstruction loss from the result. The continuous ADMM updates from NanoQuant become solutions to Sylvester equations, while its discrete projection and deployment format remain unchanged. Because the curvature is local to a given set of weights, ShamAN-Q re-measures the input curvature statistic for each layer immediately before layer factorization, periodically refreshing all statistics on the partially quantized model. ShamAN-Q also redistributes the uniform rank from NanoQuant across layers at the same total number of bits. On Qwen3-Base, ShamAN-Q lowers WikiText-2 perplexity at approx1 bpw from 27.56 to 22.96 (0.6B), 19.21 to 16.72 (1.7B), and 14.29 to 13.80 (4B) while matching or improving zero-shot accuracy on the Eleuther LM Evaluation Harness. On 0.6B, ShamAN-Q at approx0.8 bpw matches the published perplexity of NanoQuant at approx1.0 bpw.

Related

• Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basin
• G^2PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation
• ReSpinQuant: Efficient Layer-Wise LLM Quantization via Subspace Residual Rotation Approximation
• MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization

Source: arXiv cs.AI | 2026-10-01