Adaptive Inference Batching using Policy Gradients
DGX agentarXiv:2607.05272v1 Announce Type: cross Abstract: Inference serving systems must balance throughput and latency under bursty, heterogeneous workloads, yet the industry standard remains static batching