Study setting

Derivation data came from the FEAST trial which took place in six centres (both large regional referral hospitals and small district hospitals) across three countries (Kenya, Uganda, and Tanzania) from 2009 to 2011 and enrolled 3,170 sick febrile children aged between 2 months and 12 years with clinical evidence of impaired perfusion ([20], ISCRTN 69856593). FEAST was conducted in malaria endemic areas where national vaccination programmes included Haemophilus influenza type B vaccine, but not a pneumococcal vaccine. Prior and during the trial, admitting clinicians and nurses received Emergency Triage Assessment and Treatment training [21], which included the assessment of clinical features of shock. Eligible children had an abnormal temperature (pyrexia (≥37.5 °C) or hypothermia (<36 °C)), severe illness (presence of one or both of impaired consciousness (prostration, the inability of a child older than 8 months of age to sit upright or the inability of a child 8 months of age or younger to breast-feed; or coma, the inability to localize a painful stimulus) and respiratory distress) and clinical evidence of impaired perfusion (one or more of the following: capillary refill time >2 s; lower limb temperature gradient, defined as a notable temperature change from cold (dorsum of foot) to warm (knee) when running the back of hand from the toe to the knee; weak radial pulse or severe tachycardia, defined as heart rate >180 beats per min (bpm) for children <1 year old, >160 bpm for those 1 to 4 years old, >140 bpm for those ≥5 years old). Children with severe malnutrition, burns, trauma, gastroenteritis, or a presumed non-infectious cause of severe illness were excluded. Children were randomised to receive boluses of 20–40 mL/kg of 5 % human albumin solution or 0.9 % saline solution over one hour, or maintenance fluids only at 4 mL/kg/h (no bolus control group). Those with severe hypotension (systolic blood pressure <50 mmHg for those aged <1 year, <60 mmHg for those 1–4 years old, <70 mmHg for those ≥5 years old) were randomly assigned in a separate stratum to receive 40 mL/kg bolus of either albumin or saline. All children enrolled in both strata were included in this study. Standardised case report forms were completed at enrolment and at specific time points during the first 48 h. At enrolment, lactate, haemoglobin, oxygen saturation, and glucose were measured and an HIV antibody test and rapid diagnostic test for malaria were performed. An automated handheld blood analyser (i-STAT, Abbott Laboratories, Abbott Park, IL) was used for immediate analyses of pH level, potassium, base excess, blood urea nitrogen (BUN), sodium, chloride, TCO₂, and PCO₂. Children with haemoglobin <5 g/dL were routinely transfused according to national guidelines [22].

The validation data came from one of the FEAST trial sites, a rural district hospital in Kilifi, Kenya, which has a general paediatric ward and a high dependency ward. The Kenya Medical Research Institute Programme has established ward surveillance and used standardised forms to systematically collect clinical admission data on all infants and children entering the hospital wards since 1989, which has been linked to demographic surveillance in the district since 2002 [23]. Children were routinely transferred to the high dependency unit if they had impaired consciousness (prostration or coma) or deep-breathing (a clinical sign of metabolic acidosis), or if they required close medical supervision for life threatening complications such as status epilepticus, severe forms of shock, or a cardio-respiratory arrest. At admission to the high dependency unit (HDU) an extended set of clinical details were routinely collected.

The first validation datasets included children aged between 2 months and 12 years admitted to the general paediatric ward between March 2011 and December 2012 (5,173 children), and the second dataset is a subset of the first and includes all children contemporaneously admitted from the general ward to the HDU (1058/5173 children). These datasets did not include children from the FEAST trial, which finished enrolment at this centre in January 2011 and included information on the date, but not time, of death.

Other published paediatric risk scores were evaluated in the FEAST derivation and validation datasets. PRISM III was developed in paediatric intensive care units in the USA and has been validated in a variety of settings [10, 24–28]. The Bedside Pediatric Early Warning System score (PEWS) was developed in Canada to quantify severity of children in hospitalised children and help with referral to critical care experts [15]. For African paediatric populations, the AQUAMAT (African Quinine Artesunate Malaria Trial) prognostic score (0–5) was developed in a post hoc analysis from the trial dataset involving nine African countries as part of the AQUAMAT trial comparing anti-malarial treatments in children with severe malaria and included five parameters (base deficit, impaired consciousness, convulsions, elevated blood urea, and underlying chronic illness) which were independently associated with death [17, 29]. The Lamberéné Organ Dysfunction Score (LODS) was created using data from six African countries in children with malaria using only three parameters (deep breathing, coma, and prostration) [16, 30]. Berkley et al. [31] used Kilifi admission data from 1998 to 2001 to develop prognostic scores for deaths at different time points following admission, subsequently named during a published validation as the Pediatric Early Death Index for Africa (PEDIA). The AQUAMAT score has not been subject to external validation to date and PEDIA along with LODS have only recently been externally validated in Uganda in children with malaria and non-malarial illnesses [30].

Statistical analyses

A clinical bedside score (the FEAST Paediatric Emergency Triage (PET) score) was created by categorising the continuous variables using appropriate clinical cut-offs to use alongside already categorised variables in a Cox regression model. Coefficients for the categories of each variable in the model were then divided by the coefficient nearest zero and rounded to the nearest integer giving an initial score value [19]. These initial score values were then further modified to ensure a straightforward scale from 1–10 by assigning 2 to the initial value if it was >3, and 1 if it was ≤3, and dropping variables that added the least predictive ability to the model (assessed by using the Net Reclassification Index (NRI) [35]). A low score on this scale then indicated a low risk of mortality and a high score indicated a high risk of mortality.

The FEAST PET score was applied to the two validation datasets using the non-parametric area under the receiver operating curve (AUROC) to measure discriminative ability. Mortality was defined as death within 2 days of admission as time of death was not available in the two validation datasets. The FEAST data and two validation datasets were also used to validate other previously published scores. To validate the PEDIA score, immediate death (death within 4 h after admission, and calculated exactly in FEAST) was interpreted as death on the same day as admission, early death (death between 4 and 48 h) was interpreted as death within 2 calendar days of admission but not the same day, and late death (>48 h) as death occurring more than 2 days after admission. Calibration was measured by Hosmer-Lemeshow goodness-of-fit χ² tests evaluated on groups defined by quintiles [36]. PRISM III, Bedside PEWS, AQUAMAT, and PEDIA scores were calculated using the available admission variables and unavailable variables in the scores were set to 0 (as recommended). Assessments at later time points were not available to use for PRISM III, although this score recommends using the worst clinical measurement in the first 24 h [13, 27].

We also considered whether laboratory candidate predictors (Table 1; with >5 % missing data) could improve the discriminatory ability of the score in situations where they could feasibly be measured (e.g. specific research studies). Multiple imputation by chained equations under the missing at random assumption, with predictive mean matching, was therefore used for imputation, including all factors in Table 1 in the imputation model and creating 25 imputed datasets [37]. Imputed and observed values were compared visually. The NRI [35] was calculated within each imputed dataset using mortality risk cut-offs at 5 %, 10 %, and 15 %, and the range and mean of this measure across the 25 imputed datasets was used to assess whether the additional laboratory variables could be usefully added to the clinical bedside variables already included in the score. The NRI assessed the ability of each additional variable to directly increase the discriminative ability of the model by looking at risk classification categories (with an increased NRI showing more children correctly classified). Backwards elimination (exit threshold mean P = 0.05 calculated from all imputed datasets) including all laboratory markers was then used to identify the laboratory variables with the largest NRIs across the imputed datasets. These were added to the clinical prognostic model to develop an extended score including laboratory markers identified as adding important information to risk scoring by the NRI. Rubin’s rules [38] were used to combine AUROCs from the multiply imputed datasets to validate the score including laboratory markers in the FEAST control arm data [39]. Finally, in an additional analysis, Cox regression was used to identify the best prognostic model for mortality based on best subsets regression in complete cases including all laboratory markers with <10 % missing data and considering all interactions. Statistical analyses were carried out in Stata (version 13.1).