Revisiting Error Feedback for Model Quantization

arXiv:2610.18139v1 Announce Type: new Abstract: We study causal error-feedback coefficients as design variables for model quantization. Modeling instantaneous quantization errors as independent, zero-mean variables with a common fixed variance, we formulated a metric-dependent trace problem for designing feedback coefficients. The formulation resembles fixed-order GPTQ/LDLQ feedback and supports Jacobian-aware designs that account for downstream error propagation. For a positive-definite metric oldsymbol{H}, we prove that the joint rotation-feedback trace minimum is net(oldsymbol{H})^{1/n} and present an attaining pair using the geometric mean decomposition. Experiments on synthetic networks, trained MLPs, and language models show benefits from downstream-sensitive feedback, but also reveal that trace optimality need not improve finite-level performance. Small QAT experiments further illustrate direct loss-driven feedback tuning with fixed or trainable network weights.

Source: arXiv cs.LG | 2026-10-14