Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving

arXiv:2610.00499v1 Announce Type: new Abstract: As diffusion large language models (dLLMs) become more capable, they are moving from research settings to real-world extit{serving}, where request management (such as scheduling and resource allocation) relies on accurate estimation of per-request inference cost. However, common cost proxies fall short for dLLMs: output length ignores that one forward pass can unmask multiple tokens, and denoising-step count ignores the extit{heterogeneous} per-step costs. We observe that the block-autoregressive generation mechanism induces a two-dimensional execution structure over output blocks and within-block denoising steps, whereas these proxies collapse it into a scalar, discarding information essential for characterizing the cost. Motivated by this insight, we propose the Denoising Workload Surface (DWS), which preserves this two-dimensional block-step structure as a probability surface to weight the heterogeneous per-step costs. We then design a coarse-to-fine training scheme that enables a lightweight prompt-only predictor to accurately predict the complex DWS. This predictor runs efficiently even on a single CPU core, avoiding GPU contention with the serving model. Since DWS decouples request-dependent execution behavior from deployment-specific cost factors, the predictor transfers across hardware configurations without retraining. In extit{real-world} serving experiments, DWS reduces cost-prediction error by up to 2.50imes over scalar-based predictors, while the DWS-guided shortest-job-first scheduler reduces end-to-end latency by up to 1.92imes for online chatbots.

Related

• Co-occurrence Patterns of LoRA Adapters in Production Diffusion Model Inference Services
• GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inference

Source: arXiv cs.LG | 2026-10-02