Backward Conformal Prediction under Covariate Shift

arXiv:2610.16048v1 Announce Type: new Abstract: Backward conformal prediction (BCP) fixes the size of a prediction set and estimates its miscoverage, but assumes calibration and test data are exchangeable. We extend BCP to covariate shift through importance weighting, preserving its guarantees when the density ratio is known. We show that BCP's tight miscoverage report depends only on the relative weight assigned to missed calibration examples, explaining why a better domain classifier can sometimes produce a worse estimate. Motivated by the fact that BCP depends only on nonconformity scores, we propose estimating density ratios on sorted score vectors rather than high-dimensional inputs. We prove that this approach recovers the same report asymptotically under an interpretable condition and characterize the resulting bias otherwise. Experiments with simulated and benchmark distribution shifts show that the proposed method improves BCP's miscoverage estimates, while score-based weighting consistently yields more accurate estimates than embedding-based weighting.

Related

• Coverage Guarantees for Pseudo-Calibrated Conformal Prediction under Distribution Shift
• Cross-Material Support Transfer for Core-Loss Prediction Under Waveform Covariate Shift
• PIcsC: Partitioning-Induced Covariate Shift Correction
• Generalization Measures under Controlled Covariate Shift: A Regime-Aware Benchmark

Source: arXiv cs.LG | 2026-10-14