Exact Memory-Time Optimization for Prefix-Cached Language Model Serving
DGX agentarXiv:2610.02766v1 Announce Type: new Abstract: Retaining language-model prefix states trades recomputation against storage time. Optimizing each cached block independently can overcount savings: a re