Adaptive Calibration in Non-Stationary Environments

arXiv:2605.11490v1 Announce Type: new Abstract: Making calibrated online predictions is a central challenge in modern AI systems. Much of the existing literature focuses on fully adversarial environments where outcomes may be arbitrary, leading to conservative algorithms that can perform suboptimally in more benign settings, such as when outcomes are nearly stationary. This gap raises a natural question: can we design online prediction algorithms whose calibration error automatically adapts to the degree of non-stationarity in the environment, smoothly interpolating between i.i.d. and adversarial regimes? We answer this question in the affirmative and develop a suite of algorithms that achieve adaptive calibration guarantees under multiple calibration measures. Specifically, with T being the number of rounds and Cin[0,T] being an unknown non-stationary measure defined as the minimal ell_1 deviation of the mean outcomes, our algorithms attain widetilde{O}(sqrt{T}+(TC)^{frac{1}{3}}) for ell_1 calibration error and widetilde{O}((1+C)^{frac{1}{3}}) for both ell_2 and pseudo KL calibration error. These bounds match the optimal rates in the stationary case (C=0) and recover known guarantees in the fully adversarial regime (C=T). Our approach builds on and extends prior work [Hu et al., 2026, Luo et al., 2025], introducing an epoch-based scheduling together with a novel non-uniform partition of the prediction space that allocates finer resolution near the underlying ground truth.

Source: arXiv cs.LG | 2026-05-13