Births are difficult to predict even with rich survey and full-population register data
Elizaveta Sivak, Emily M. Cantrell, Thomas Emery, Javier Garcia-Bernardo, Flavio Hafner, Kasia Karpinska, Malte Lüken, Adrienne Mendrik, et al.
cs.LG
Quick verdict
147-researcher challenge finds births hard to predict (best F1 0.59/0.76 register/survey), below a simulated chance ceiling; rigorous, assumption-sensitive.
Summary
The paper reports PreFer ('Predicting Fertility in the Netherlands'), a two-phase data challenge in which 147 researchers predicted whether Dutch residents aged 18–45 had a child between 2021 and 2023 using data through 2020. Phase 1 used LISS survey data (69 valid submissions from 41 teams; 1,382 respondents with known outcomes); phase 2 used full-population registers (~5.88 million people; 11 submissions from 12 teams). Best F1 was 0.59 (register, 95% CI [0.589, 0.593]) and 0.76 (survey, CI [0.68, 0.83]); gradient-boosted tree ensembles won both phases, while a life-course transformer and a fine-tuned LLM reached only 0.57. Survey models improved little over a fertility-intentions baseline (F1 0.64), and ablations show intentions are critical (0.76→0.65 without). After prevalence matching (22% vs 15%), survey performance (0.63–0.68) still exceeded register performance. Via Monte Carlo simulation of fecundability, fetal survival, and contraceptive failure, the authors estimate a predictive ceiling of F1 ≈ 0.86–0.94 (survey) and 0.88–0.96 (register), depending on the assumed contraceptive-failure birth share (1–17%). Observed performance falls short of this ceiling, implicating data/method limits plus unmodelled chance. Ensembling did not help; learning curves plateau near 100k (register) and 1k (survey) observations. Limitations include the small survey holdout (n=395), an assumption-sensitive ceiling, register access limited to 12 largely pre-selected teams, and panel attrition restricting outcome ascertainment to ~20% of the LISS target sample.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Births within three years are only moderately predictable even under near-ideal conditions: best F1 0.59 from full-population registers and 0.76 from the survey, versus a simulated biological ceiling of ≈0.86–0.96.
- Advanced sequence architectures (a life-course transformer and a fine-tuned LLM, both F1 = 0.57) do not outperform gradient-boosted tree ensembles.
- Full-population register data does not outperform a ~1,400-person survey once prevalence is matched (0.59 vs 0.63–0.68), and register gains saturate beyond ~100k training observations.
- Stochasticity in conception, fetal survival, and contraceptive failure alone imposes a non-trivial predictability ceiling, so part of predictive error is irreducible in principle.
Strengths
Unusually rigorous challenge design
Two-phase PreFer with household-level train/holdout splits (register: 70/30 of 5,878,530; survey: 395-person holdout), pre-2021-only features, withheld outcomes, capped submissions (5 survey rounds, 1 register), and Docker-containerized survey submissions re-run by organisers — leakage control far beyond typical prediction papers.
Original, stress-tested predictive ceiling
The Monte Carlo simulation (Methods; SI 2.6) draws fecundability and age at sterility from empirical distributions, adds fetal survival and 1–17% contraceptive-failure births, and yields a ceiling of F1 0.86–0.94 (survey) / 0.88–0.96 (register); a random-forest robustness check reproduces the logistic-regression result.
Careful cross-source comparison
Meaningful baselines and ablations
Transparency and reproducibility
Weaknesses
Assumption-dependent ceiling
Small survey holdout limits ranking claims
Severe survey selection via attrition
Register-phase breadth is limited
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Review: Births are difficult to predict even with rich survey and full-population register data (arXiv 2609.01194)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
How accurately can anyone predict who will have a child?
In this study, the best models reached an F1 of 0.76 using survey data and 0.59 using full-population register data for predicting a birth within three years. The best survey model correctly identified 66% of people who had a child, with 91% of its birth predictions being correct.
Why are births so hard to predict?
The authors simulate the chance inherent in reproduction — monthly conception probabilities, pregnancy loss, and contraceptive failure — and show this randomness alone caps predictive accuracy at roughly F1 0.86–0.96. Observed models fall below that ceiling, meaning limits of current data and methods plus unmodelled chance both contribute.
Did AI and deep learning beat simpler models at predicting births?
No. Gradient-boosted decision tree ensembles won both phases of the challenge. A transformer trained on life-course sequences and a fine-tuned large language model both scored slightly lower (F1 0.57 vs 0.59) on register data.
Is bigger data better for predicting life outcomes?
Not automatically in this study: the full-population register (millions of people) did not beat a ~1,400-person survey once outcome prevalence was matched, and learning curves show gains from sample size alone become marginal beyond about 100,000 register observations or about 1,000 survey observations.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The review is based on a sampled excerpt: several SI segments marked as sampling gaps (including the SI 2.11 leakage evaluation, SI 2.12, and portions of the submission-description tables) were not visible; judgments about those parts rely on main-text references only.
- Figures were not available in the extraction; figure-based results are judged from captions and surrounding text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.