A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work

arXiv:2604.18555v1 Announce Type: new Abstract: This note clarifies the relationship between the recent TurboQuant work and the earlier DRIVE (NeurIPS 2021) and EDEN (ICML 2022) schemes. DRIVE is a 1-bit quantizer that EDEN extended to any b>0 bits per coordinate; we refer to them collectively as EDEN. First, TurboQuant_{ext{mse}} is a special case of EDEN obtained by fixing EDEN's scalar scale parameter to S=1. EDEN supports both biased and unbiased quantization, each optimized by a different S (chosen via methods described in the EDEN works). The fixed choice S=1 used by TurboQuant is generally suboptimal, although the optimal S for biased EDEN converges to 1 as the dimension grows; accordingly TurboQuant_{ext{mse}} approaches EDEN's behavior for large d. Second, TurboQuant_{ext{prod}} combines a biased (b-1)-bit EDEN step with an unbiased 1-bit QJL quantization of the residual. It is suboptimal in three ways: (1) its (b-1)-bit step uses the suboptimal S=1; (2) its 1-bit unbiased residual quantization has worse MSE than (unbiased) 1-bit EDEN; (3) chaining a biased (b-1)-bit step with a 1-bit unbiased residual step is inferior to unbiasedly quantizing the input directly with b-bit EDEN. Third, some of the analysis in the TurboQuant work mirrors that of the EDEN works: both exploit the connection between random rotations and the shifted Beta distribution, use the Lloyd-Max algorithm, and note that Randomized Hadamard Transforms can replace uniform random rotations. Experiments support these claims: biased EDEN (with optimized S) is more accurate than TurboQuant_{ext{mse}}, and unbiased EDEN is markedly more accurate than TurboQuant_{ext{prod}}, often by more than a bit (e.g., 2-bit EDEN beats 3-bit TurboQuant_{ext{prod}}). We also repeat all accuracy experiments from the TurboQuant paper, showing that EDEN outperforms it in every setup we have tried.

Source: arXiv cs.LG | 2026-04-21