Activation Sparsity with Weight Approximation for Faster LLM Decoding on Offloaded Weights
arXiv:2610.02598v1 Announce Type: new Abstract: Deploying LLMs on consumer-grade GPUs with insufficient memory to hold their weights can result in prohibitively slow inference, because decoding repeat