Fig. 7

Preterm birth prediction models accurately generalize to an independent cohort. Performance of preterm birth prediction models trained at Vanderbilt applied to UCSF cohort. Models were trained using ICD-9 codes present before 28 weeks of gestation at Vanderbilt on 16,857 women and evaluated on a held-out set at Vanderbilt (n = 4215, gold) and UCSF cohort (n = 5978, blue). A Models accurately predicted preterm birth at Vanderbilt (ROC-AUC = 0.72) and at UCSF (ROC-AUC = 0.80). The higher ROC-AUC at UCSF is driven by the lower prevalence of preterm birth in this cohort. B Models performed better than baseline prevalence (chance) based on the precision-recall curve at Vanderbilt (PR-AUC = 0.34) and at UCSF (PR-AUC = 0.31). Note that in contrast to the models presented previously, this one was trained only on ICD-9 codes, due to the lack of CPT codes in the UCSF cohort. C The top 15 features with the highest mean absolute SHAP score in the Vanderbilt cohort (gold square) or UCSF cohort (blue circle). The majority of the features were shared across cohorts and captured known risk factors (fetal abnormalities, history of preterm birth, etc.), pregnancy screening visits, and supervision of high-risk pregnancies. Feature importance estimates were strongly correlated between the two cohorts (Additional file 1: Fig. S10). Cohort demographics are given in Table 1