Fairness under Prediction Error in Predict-then-Optimize Pipelines

arXiv:2610.18593v1 Announce Type: new Abstract: Fairness constraints in predict-then-optimize pipelines are necessarily enforced on predicted outcomes hat{y}, since true outcomes y are unobserved at decision time. We study whether a demographic parity constraint enforced this way continues to hold against y, and how the resulting violation scales with prediction error. Using a budget-constrained allocation problem solved as a linear program, we show empirically that the true-outcome fairness gap grows with prediction error sigma and saturates, well fit by c(1 - e^{-sigma/au}) with c = 0.1545 pm 0.0103 and au = 3.9045 pm 0.3745; its mean is invariant to sample size while its variance decays at the standard O(1/sqrt{n}) rate. We derive a two-part theoretical bound: a closed-form local sensitivity result showing the gap's first-order response to prediction error depends on only two individuals per group, validated against the solver to within three significant figures; and a universal, error-independent ceiling following from the boundedness of the decision variable, confirmed consistent with but looser than the empirical saturation level. On US Census income data, enforcing demographic parity on a trained predictor's output reduces an unconstrained disparity in true income to roughly one-third of its original size, leaving 34.4% pm 3.3% (California) and 35.7% pm 1.8% (Texas) of the disparity intact and undetectable from the predictions alone -- consistent not only in mechanism but in approximate magnitude across two states with different demographics and income distributions, and stable across ten independent train/test splits in each. A fairness constraint that appears fully satisfied at decision time can mask a substantial, systematic violation of the outcome it was meant to protect.

Source: arXiv cs.LG | 2026-10-14