BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference
arXiv:2609.13592v1 Announce Type: cross Abstract: GPU memory bandwidth and capacity limit throughput in large language model (LLM) inference. The GPU memory system consists of a primary tier of high-b