Research
A Target-Centric Survey of Quantization-Aware Training
arXiv:2608.29667v1 Announce Type: new Abstract: The rapid development of LLMs incurs prohibitive memory footprints and intensive computational demands. Quantization-Aware Training (QAT) techniques hav
arXiv:2608.29667v1 Announce Type: new Abstract: The rapid development of LLMs incurs prohibitive memory footprints and intensive computational demands. Quantization-Aware Training (QAT) techniques have emerged as a promising solution to address these challenges by explicitly simulating quantization effects during model training, yielding low-bit models that achieve accuracy comparable to their full-precision counterparts. In this work, we provide a target-centric survey of QAT, aimed at clarifying both its theoretical foundations and its evolving implementation landscape. We systematically review existing QAT methods through a target-centric taxonomy and synthesize cross-target differences in error characteristics, numerical formats, and strategy transferability. We further summarize QAT evaluation paradigms and discuss challenges in optimization and deployment, outlining potential directions for future research.
Related
- Activation Outliers Matter: Robust Recovery for Quantized Multimodal LLMs
- Inner Product Aware Quantization: Provably Fast, Accurate, and Adaptive Algorithms
- A Controlled Diagnostic Study of Hardware-Induced Distortions in Hardware-Aware Training
- OASIS: Online Activation Subspace Learning for Memory-Efficient Training
Source: arXiv cs.LG | 2026-09-01