CATS: Cascaded Adaptive Tree Speculation for Memory-Limited LLM Inference Acceleration
DGX agentarXiv:2605.11186v1 Announce Type: new Abstract: Auto-regressive decoding in Large Language Models (LLMs) is inherently memory-bound: every generation step requires loading the model weights and interm