Anytime and Difficulty-Adaptive PAC-Bayes for Constrained Density-Ratio Network with Continual Learning Guarantees

arXiv:2605.17212v1 Announce Type: new Abstract: A unified framework for learning under covariate shift is presented, in which a constrained density-ratio network approximates the Radon-Nikodym derivative r^star = dP/dQ from source Q to target P, supports an importance-weighted empirical risk, and feeds an anytime PAC-Bayes generalization certificate. A change-of-measure identity decomposes the gap between target risk and importance-weighted source risk into a ratio-bias term, controlled by the L^2(Q) closeness of the learned ratio to r^star, and a generalization-gap term, controlled by the variability of the weighted loss. Three structural identities of a Radon-Nikodym derivative, normalization, moment matching, and a second-moment penalty controlling the effective sample size, are imposed as hard integral constraints through an augmented-Lagrangian scheme. PAC-Bayes is then instantiated on the weighted risk in a fixed-time regime that yields Bernoulli-KL bounds, a KL-regularized objective whose minimizer is the network-weighted Gibbs posterior, and a stability statement on L^2(Q) perturbations of the learned ratio, and in an anytime regime that builds a time-uniform certificate by geometric peeling across epochs. A pre-registered two-campaign protocol combining a patch test against analytic ground truth with a real-data deployment under intrinsic distribution shift validates the framework. The network produces a calibrated covariate ratio on real data, reduces the target 0/1 loss relative to unweighted empirical risk minimization and to classical direct ratio-estimation baselines, and attains the anytime certificate as the construction promises. A single pre-registered failure of the fixed-time coverage claim is recorded, with per-split coverage aligning one-to-one with the magnitude of the label shift, confirming that the covariate-only assumption is operationally tight rather than a defect of the certificate.

Source: arXiv cs.LG | 2026-05-19