Research
Efficient VQ-QAT and Mixed Vector/Linear quantized Neural Networks
arXiv:2604.23172v1 Announce Type: new Abstract: In this work, we developed and tested 3 techniques for vector quantization (VQ) based model weight compression. To mitigate codebook collapse and enable
arXiv:2604.23172v1 Announce Type: new Abstract: In this work, we developed and tested 3 techniques for vector quantization (VQ) based model weight compression. To mitigate codebook collapse and enable end-to-end training, we adopted cosine similarity-based assignment. Building on ideas from attention-based formulations in Differentiable K-Means (DKM), we further improved this approach by using cosine similarity for assignment combined with top-1 sampling and a straight-through estimator, thereby eliminating the need for weighted-average reconstruction. Finally, we investigated the use of differentiable neural architecture search (NAS) to adaptively select layer-wise quantization configurations, further optimizing the compression process. Although our method does not consistently outperform existing approaches across all quantization levels, it provides useful insights into the design trade-offs and behaviors of VQ-based model compression methods.
Related
- DPQuant: Efficient and Differentially-Private Model Training via Dynamic Quantization Scheduling
- From 2:4 to 8:16 sparsity patterns in LLMs for Outliers and Weights with Variance Correction
- Resource-aware Mixed-precision Quantization for Enhancing Deployability of Transformers for Time-series Forecasting on Embedded FPGAs
Source: arXiv cs.LG | 2026-04-28