Research
When Does Dynamic Ensembling Pay Off? Diagnosing Regionwise Gains in Regression under Distribution Shift
arXiv:2608.18330v1 Announce Type: new Abstract: Whether input-dependent ('dynamic') combination of a regression model pool beats the best static blend depends on the shift and is rarely known before d
arXiv:2608.18330v1 Announce Type: new Abstract: Whether input-dependent ("dynamic") combination of a regression model pool beats the best static blend depends on the shift and is rarely known before deployment. Can a small labeled target-domain probe tell us when reallocating trust across regions of the input space will pay off? We answer this with widehat{D}{CF5}, which estimates from the probe the cross-fitted gain of the regionwise convex combination over the best static convex blend: the realizable value of deciding, region by region, whom to trust. Across a frozen suite of 12 dataset-shift pairs (spatial, temporal, domain, feature-cluster), widehat{D}{CF5} predicts realized regionwise test gains with dataset-level Spearman +0.98 (95% CI [+0.83, +1.00]; p=5imes10^{-5}), including two cases overturning preregistered expectations. The relationship holds in a 16-pair sensitivity analysis (Spearman +0.83), whereas alternative probe diagnostics reach at most +0.66. This contrast isolates regional trust reallocation: correlation is +0.98 for regionwise-convex gain, but +0.01 for smooth covariate-dependent stacking after affine correction. A controlled generator shows dynamic gains arise from the interaction of shift heterogeneity and local competence, increase with shift severity, and become realizable between 128 and 256 probe labels in the tested grid. The Probe-Validated Ensemble Selector chooses among a static affine stacker and dynamic realizers, deploying a candidate only when a held-out lower confidence bound clears the static-convex floor. In a preregistered prospective batch, it matched or improved the floor in all 12 runs; two deployments reduced test risk by 11% and 16%, while the gate rejected a candidate whose un-gated deployment incurred >30imes the static loss. We release OpenRegShift, a reproducible evaluation harness for regression ensembles under distribution shift.
Related
- From Uncertainty to Failure Attribution: Self-Diagnosing Models for Failure Attribution under Distribution Shift
- Adaptive deep nonparametric regression from dependent data under covariate shift
- Dynamic Regret for Online Regression in RKHS via Discounted VAW and Subspace Approximation
Source: arXiv cs.LG | 2026-08-20