Births are difficult to predict even with rich survey and full-population register data

arXiv:2609.01194v1 Announce Type: new Abstract: Major life events have proven difficult to predict. Does this reflect limits of theory, data, and algorithms, or the large role of chance? We examine one outcome - having a child within three years - through a near-ideal setting for prediction: a data challenge where 147 researchers predicted births for Dutch residents aged 18-45, using survey data and full-population registers. Methods ranged from logistic regression to a large language model and transformers. Predictions were moderately accurate (best F1: register 0.59, survey 0.76); advanced models did not outperform classical ones; and the larger registers did not beat the survey. Simulating the stochastic biology of conception and pregnancy, we estimated a predictive ceiling (survey F1 ~ 0.86-0.94, register 0.88-0.96). Observed performance falls short of this ceiling, implicating imperfect data, methods, and unmodelled chance, while the ceiling itself shows that chance in reproduction alone sets a non-trivial limit on predicting individual lives.

Related

• An Objective Performance Evaluation of the LSTM Networks in Time Series Classification
• How well does Classification Accuracy capture Concept Drift Detection Quality? An overview of Concept Drift Detection evaluation
• Information-theoretic Limits of Learning and Estimation
• Scalable Counterfactual Risk Estimation for Rare Events in Longitudinal Data

Source: arXiv cs.LG | 2026-09-02