From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix

arXiv:2609.01572v1 Announce Type: new Abstract: Data-residency constraints force enterprises to self-host LLMs, but continuous adoption of newer models without decommissioning their predecessors expands the serving fleet, fragmenting a finite GPU pool. We consolidate traffic from over 200 internal applications onto a single model by closing quality gaps identified through production error analysis along three axes: instruction following, function-calling, and internal task distribution. Quality is tracked by offline benchmarks stratified to production traffic and scored by deterministic verifiers or calibrated LLM judges. Rather than optimising all objectives jointly, which introduces cross-domain reward interference, we train a separate GRPO expert per axis and merge them via two-stage SLERP. Each expert's reward exposes a distinct failure mode, namely semantic collapse, over-calling, and verbosity hacking, each requiring a domain-specific fix. In non-reasoning mode the recipe surpasses a {sim}7imes larger by total parameters baseline on the in-house Arena with 69.6 to 65.8, instruction following with 0.85 to 0.83, and function-calling with 0.79 to 0.77, while lifting general dialogue benchmarks. The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.

Related

• AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
• PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoning
• Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs

Source: arXiv cs.CL | 2026-09-02