LeanStream: A Speculate-and-Refine Streaming Framework for Efficient on-Device LLM Inference
DGX agentarXiv:2609.03079v1 Announce Type: new Abstract: On-device LLM inference is attractive for privacy and responsiveness, but remains challenging on mobile and embedded devices because model weights far e