When Words Predict Workload

arXiv:2607.04951v1 Announce Type: cross Abstract: Standard distributed ac{llm} schedulers rely on static token counts or rolling latency averages, making them susceptible to failures on statutorily constrained text. On ac{epo} claims governed by Article 84 ac{epc}, linguistic rigidity makes human and machine authorship statistically indistinguishable. Resolving this ambiguity mid-flight forces dynamic multi-model ensemble expansion, triggering unpredictable KV-cache and weight-allocation spikes that saturate consumer-grade edge GPU VRAM and cause severe ac{oom} crashes. To prevent hardware collapse, we propose a CPU-side Linguistic Resource Forecasting (LRF) gateway. The gateway extracts a 16-dimensional text-structure vector and applies an XGBoost predictor to forecast trap-band membership. The resulting escalation probability (Pesc) is evaluated against a dynamic, closed-form routing threshold (Tauroute(t)) computed via real-time latency telemetry. Requests are safely routed to either a local Qwen2.5-7B edge worker or a remote contrastive ensemble (Qwen2.5 7B + 32B) on an NVIDIA H100 before any edge GPU memory is allocated. In a 6,000-request live trial, the LRF gateway reduced the operational misroute fraction (R_{mis}) to 0.087--0.095, an order of magnitude below the token-count baseline (0.849). Peak edge VRAM remained safely bounded at SI{4.82}{gibiyte} (under the SI{8}{gibiyte} ceiling) across a 27imes variation in ac{wan} delay. The predictor achieved a live-trial AUROC of 0.84, and the dynamic Tauroute(t) controller yielded an 8.2% relative reduction in misroutes compared to an equivalent static threshold.

Source: arXiv cs.CL | 2026-07-07