A Layer Importance Metric for Quantization Accounting for the Speed-Quality Trade-off in Autoregressive Models
arXiv:2608.26926v1 Announce Type: new Abstract: Small language models (sLLMs) are nowadays hosted on devices with limited memory and computational budget. In an autoregressive setup, inference is memo