Localising Dropout Variance in Twin Networks

arXiv:2507.03622v2 Announce Type: replace Abstract: Accurate individual treatment-effect estimation demands not only reliable point predictions but also uncertainty measures that help practitioners locate the source of model failure. We introduce a layer-wise variance decomposition for deep twin-network models: by toggling Monte Carlo Dropout independently in the shared encoder and the outcome heads, we split total predictive variance into an encoder component (sigma_{enc}^2) and a head component (sigma_{head}^2), with sigma_{enc}^2 + sigma_{head}^2 approx sigma_{tot}^2 by the law of total variance. Across three synthetic covariate-shift regimes, the encoder component dominates under distributional shift (rho_{enc}=0.53) while the head component becomes informative only once encoder uncertainty is controlled. On a real-world twins cohort with induced multivariate shift, only sigma_{enc}^2 spikes on out-of-distribution samples and becomes the primary error predictor (rho_{enc}!approx!0.89), while sigma_{head}^2 remains flat. The decomposition adds negligible cost over standard MC Dropout and provides a practical diagnostic for deciding whether to collect more diverse covariates or more outcome data.

Source: arXiv cs.LG | 2026-05-13