Fig. 3
Demographic, clinical, and genetic features do not improve preterm birth prediction compared to billing codes. A Framework for evaluating change in preterm birth prediction performance after incorporating diverse types of EHR features with billing codes (ICD-9 and CPT codes). We used only features and billing codes occurring before 28 weeks of gestation. EHR features are grouped by sets of demographic factors (age and race), clinical keywords (UMLS concept unique identifiers from obstetric notes), clinical labs, and genetic risk (polygenic risk score for preterm birth). We compared three models for each feature set: (1) using only the feature set being evaluated (pink), (2) using only billing codes (“Billing codes,” purple), and (3) using the feature set combined with billing codes (“Both,” gray). For each feature set, we considered the subset of women who had at least one recorded value for the EHR feature and billing codes. All three models for a given EHR feature set considered the same pregnancies, but there are differences in the cohorts considered across the feature set due to the differences in data availability; ntotal is the number of women (training and held-out) for each feature set. B Each of the three models (x-axis) and their ROC-AUC and PR-AUC (y-axis) are shown. Each of the additional EHR features performed worse than the billing codes-only model and did not substantially improve performance when combined with the billing codes. Dotted lines represent a chance of 0.5 for ROC-AUC and the preterm birth prevalence for PR-AUC. Even when including EHR features before and after delivery in this framework revealed the same pattern with no substantial improvement in predictive performance compared to the billing codes-only model (Additional file 1: Fig. S6)