Goal-Oriented Lower-Tail Calibration of Gaussian Processes for Bayesian Optimization

arXiv:2605.20145v1 Announce Type: cross Abstract: Bayesian optimization (BO) selects evaluation points for expensive black-box objectives using Gaussian process (GP) predictive distributions. Kernel choice and hyperparameter selection can lead to miscalibrated predictive distributions and an inappropriate exploration-exploitation trade-off. For minimization, sampling criteria such as expected improvement (EI) depend on the predictive distribution below the current best value, so lower-tail miscalibration directly affects the sampling decision. This article studies goal-oriented calibration of GP predictive distributions below a low threshold t in the noiseless setting, for standard GP models with hyperparameters selected by maximum likelihood. A framework for predictive reliability below t is introduced, based on two notions of spatial calibration: occurrence calibration over the design space and thresholded mu-calibration on sublevel sets of the form {xinX, f(x)le t}. Building on this framework, we propose tcGP, a post-hoc method that calibrates GP predictive distributions below~t, and we show that the resulting EI-based global optimization algorithm remains dense in the design space. Experiments on standard benchmarks show improved lower-tail calibration and BO performance relative to standard GP models and globally calibrated GP models.

Source: arXiv cs.LG | 2026-05-20