SAGE: Pushing 2-Bit Vector-Quantized LLMs toward Real-World Usability
DGX agentarXiv:2610.14745v1 Announce Type: new Abstract: Extreme low-bit weight-only quantization is an effective approach to reducing the deployment costs of large language models. Below 3 bits, scalar quanti