External validation of the Scandinavian guidelines for management of minimal, mild and moderate head injuries in children

Background Clinical decision rules (CDRs) aid in the management of children with traumatic brain injury (TBI). Recently, the Scandinavian Neurotrauma Committee (SNC) has published practical, evidence-based guidelines for children with Glasgow Coma Scale (GCS) scores of 9–15. This study aims to validate these guidelines and to compare them with other CDRs. Methods A large prospective cohort of children (< 18 years) with TBI of all severities, from ten Australian and New Zealand hospitals, was used to assess the SNC guidelines. Firstly, a validation study was performed according to the inclusion and exclusion criteria of the SNC guideline. Secondly, we compared the accuracy of SNC, CATCH, CHALICE and PECARN CDRs in patients with GCS 13–15 only. Diagnostic accuracy was calculated for outcome measures of need for neurosurgery, clinically important TBI (ciTBI) and brain injury on CT. Results The SNC guideline could be applied to 19,007/20,137 of patients (94.4%) in the validation process. The frequency of ciTBI decreased significantly with stratification by decreasing risk according to the SNC guideline. Sensitivities for the detection of neurosurgery, ciTBI and brain injury on CT were 100.0% (95% CI 89.1–100.0; 32/32), 97.8% (94.5–99.4; 179/183) and 95% (95% CI 91.6–97.2; 262/276), respectively, with a CT/admission rate of 42% (mandatory CT rate of 5%, 18% CT or admission and 19% only admission). Four patients with ciTBI were missed; none needed specific intervention. In the homogenous comparison cohort of 18,913 children, the SNC guideline performed similar to the PECARN CDR, when compared with the other CDRs. Conclusion The SNC guideline showed a high accuracy in a large external validation cohort and compares well with published CDRs for the management of paediatric TBI.


Background
Traumatic brain injury (TBI) is a major global health problem [1] with a general incidence of 262 per 100,000 per year [2], which does not seem to be declining despite increased knowledge and prevention strategies [3]. TBI is common in both developed and also in low-and middle-income countries and is associated with considerable mortality and morbidity [3,4]. The incidence of TBI is higher in children than in adults [5], children are often more difficult to assess and neuroradiological management is associated with concerning health risks [6,7].
Initial management is focused on the detection or exclusion of significant brain injury, in particular injuries that would need neurosurgical procedures. The gold-standard investigation is computed tomography (CT), which reliably detects and excludes intracranial complications following head injury. However, the concerns of economic, logistic and radiation burden of increasing CT use limits its use for all children with head injury [8][9][10][11]. An alternative option is admission to hospital of intermediate risk groups for clinical observation with deferred CT imaging if signs and symptoms worsen or do not improve, a practice which has been demonstrated to be safe but may be associated with higher health care costs [12,13].
Clinical decision rules (CDRs) have been developed to stratify patients according to the risk of important outcomes and hence indication for CT, with the goal of optimising resource use while assuring detection of important intracranial injuries. Several CDRs for children have been developed including the Pediatric Emergency Care Applied Research Network (PECARN) rule, the Canadian Assessment of Tomography for Childhood Head Injury (CATCH) rule and the Children's Head Injury Algorithm for the Prediction of Important Clinical Events (CHALICE) rule [14][15][16]. These were derived using high-quality methods and have recently been externally validated in a large prospective cohort [17]. Although the PECARN rule seems to display the best accuracy [17], in particular a very high sensitivity for relevant outcomes, the actual impact of such a rule will depend on the target population and baseline management routines. Although not borne out by recent data [18,19], there is an ongoing concern that these rules may increase CT use in some settings [20].
Recently, the Scandinavian Neurotrauma Committee (SNC), a non-profit organisation of neurosurgeons, anaesthesiologists, intensivists, neurologists and other specialties from Sweden, Norway, Denmark, Finland and Iceland, with an interest in TBI, developed and published evidence-based guidelines for management of minimal, mild and moderate head injuries in adults [21] and children [20]. These guidelines offer a comprehensive guide to TBI management, including selection of patients for CT scan and/or hospital observation, in the context of the Scandinavian health care system, see Fig. 1. As these guidelines were not based on a derivation cohort, validation, in particular external validation, is required before widespread clinical implementation.
Recently, Babl et al. published an appropriately powered multicentre validation and comparison study, the Australian Paediatric Head Injury Rules Study (APHIRST), comparing the accuracy of the PECARN, CHALICE and CATCH CDRs [17,22]. This study included sufficient predictor variables to externally validate the SNC guidelines. In addition to an external validation as the primary aim, we set out to compare the SNC guidelines to the PECARN, CHALICE and CATCH CDRs.

Design and setting
The APHIRST study was a prospective multicentre observational study, which enrolled 20,137 children (age < 18 years) with head injury of all severities at ten Australian and New Zealand centres of the Paediatric Research in Emergency Departments International Collaborative (PREDICT) network [23]. Predictor variables from the PECARN, CATCH and CHALICE were collected, and the performance accuracy of these rules was externally validated. Detailed information on this study can be found in the primary publication [16] and the protocol publication [22].
The SNC guideline is intended for all children (< 18 years) with head injury and a GCS of 9-15, presenting within 24 h of injury [20]. Being a tool for selecting children for imaging, those children who have already had imaging are excluded.

Procedure
In most cases, the clinical predictors elicited in the APHIRST study were identical to the ones used in the SNC guideline. In the few instances where variables were different, assumptions were made a priori to analysis, see Table 1. SNC guideline parameters were applied to the APHIRST dataset, and suggested management was noted.
As with the APHIRST parent publication [17], the SNC guideline was assessed in two ways. Firstly, the cohort was inputted into the SNC guideline according to the guideline inclusion criteria and with the intended SNC primary outcomes, neurosurgical intervention and intracranial injury [20]. Secondly, the same comparison cohort used in the parent publication [17], i.e. children with a GCS of 13-15 who presented within 24 h of injury, was used in order to compare the SNC guideline with PECARN, CHALICE and CATCH CDRs. The common outcome variable used to compare the accuracy of the SNC guideline and the three CDRs was the presence of clinically important TBI (ciTBI) [14].

Definitions
Neurosurgery was defined as any neurosurgical procedure for TBI. ciTBI was defined according to the PECARN definition; death from TBI, neurosurgical intervention for TBI, intubation of more than 24 h from TBI or hospital admission of two nights or more for TBI, associated with TBI on CT [14].
TBI on CT was defined as any acute intracranial finding revealed on CT that was attributable to acute injury, including closed depressed skull fractures and pneumocephalus, but excluding non-depressed skull fractures and basilar skull fractures [14].
As the SNC guideline recommends both CT and/or hospital admission with observation, depending on the risk group, a binary variable was assumed where CT and/or observation was compared to discharge. This is similar to the method used for the external validation of the PECARN rule [17].

Analysis
We did not undertake a separate sample size calculation beyond the sample size calculation undertaken for the APHIRST parent study [22].
Sensitivity, specificity and predictive values were calculated with corresponding 95% confidence intervals. Differences between risk groups were assessed by Fisher's exact test.

Results
From the original sample of 20,137 children, we applied SNC guideline eligibility criteria and excluded 1013 children who presented > 24 h after injury and 117 with GCS < 9. Therefore, a total of 19,007 children (94% of the total cohort) were applicable to the SNC guideline. Selected patient characteristics are shown in Table 2.

Validation of SNC guideline
Thirty-two (0.17%) children needed neurosurgery, 183 (1.0%) had ciTBI, 276 (1.5%) had a TBI on CT and one patient died (TBI was not the cause of death in this patient). The distribution of children in the different SNC risk categories, with corresponding neurosurgery, ciTBI and brain injury on CT outcomes, is shown in Fig. 2. There were significant differences between the risk groups in terms of ciTBI frequency. When combining groups to represent the recommendations of 'immediate CT' , 'observation or CT' , 'observation' and 'discharge' , there were also highly significant differences, see Fig. 2. In the primary analysis of the SNC guideline, point sensitivities for the detection of neurosurgery, ciTBI and TBI on CT were 100, 98 and 95%, respectively, and point specificities were 58, 59 and 59%, respectively (Table 3).

SNC guideline comparison with CDRs
Of 18,913 children included in the comparison cohort, we further omitted patients with a GCS of 9-12. Twenty-four (0.13%) children needed neurosurgery, 160 (0.85%) had ciTBI, 251 (1.3%) had TBI on CT and one patient died. Point sensitivities and specificities for the Fig. 1 The Scandinavian Neurotrauma Committee (SNC) guideline for management of children (< 18 years) with minimal, mild and moderate traumatic brain injury (TBI) [20]. GCS Glasgow Come Scale, LOC loss of consciousness, CT computed tomography. Modified from Astrand et al. [20] detection of neurosurgery, ciTBI and TBI on CT were similar to the validation cohort (Table 4). Four patients with ciTBI and 14 with TBI on CT were missed by the SNC guideline ( Table 5). All the missed ciTBIs were classified as such due to admission to hospital > 2 days for TBI, with none needing any specific intervention.

CT and observation rate
Applying the SNC guideline would have resulted in a CT/in-hospital observation rate of 42% in both the validation sample and in the comparison cohort. When strictly applied, the mandatory CT rate for the SNC guideline ( Fig. 1) would have been only 5% in both the validation and comparison cohorts, with an 18% rate of observation or CT and a 19% rate for only observation (no CT). If children with multiple risk factors and medium-risk factors (observation or CT according to the guideline) were all to receive a CT, the rate would be 23%.

Discussion
In this study, we were able to apply a multinational clinical head injury guideline from Scandinavia to a large, previously collected data set of head injured children and externally assess the accuracy of the guideline. This study appears to adequately validate the accuracy of the SNC guidelines for the management of minimal, mild and moderate head injury in children. In the validation cohort, the guideline displayed a high sensitivity for important outcomes, missing four patients with ciTBI, 14 patients with TBI on CT scan, but no patients requiring neurosurgery out of over 19,000 patients. The SNC guideline was designed to be a pragmatic and universal aid [20]; as demonstrated by the large number of patients, the guideline could be applied to the current APHIRST cohort. Only patients with severe head injury, those who already had neuroimaging and those seeking medical care after 24 h are excluded by the guideline.
When comparing the applicability of the SNC guideline with well-known CDRs, when used as designed [24],  [17]. Adherence to clinical guidelines and CDRs may be problematic [25,26], especially when dealing with specific and multiple inclusion criteria for guideline applicability [24]. A pragmatic guideline with near-universal inclusion is therefore desirable to ensure clinical use as intended.
Comparing guidelines is difficult due to the differing inclusion criteria, clinical predictors and outcome variables used. Using the APHIRST dataset, a comparison cohort (identical to the SNC inclusion criteria with the exception of patients with GCS 9-12) could be used to directly compare the accuracy of the different rules. The performance of the SNC guideline was similar to the PECARN CDR (high sensitivity, lower specificity) rather than the CATCH and CHALICE CDRs (lower sensitivities but higher specificities). However, the confidence intervals overlap, meaning a statistical difference cannot formally be established. Nonetheless, for the outcome of neurosurgery (the primary outcome variable of the SNC guideline and arguably the most important outcome variable in TBI [17,20]), the SNC guideline was 100% sensitive, with a relatively high lower 95% confidence interval (85.8%) and a higher overall specificity than PECARN. As an evidence-based guideline, the largest individual evidence contributor for the synthesis of the SNC guideline was derived from the PECARN study, which likely explains the similarities in performance. The projected CT or admission rate for SNC of 42%, in both the validation and comparison sample, is difficult to compare with the CATCH or CHALICE CDRs. Both dichotomise patients into CT/no CT, without consideration of observation, with projected CT rates for CATCH of 30% (using all predictors) and for the CHAL-ICE rule of 22%, for the comparison cohorts. However, as with the PECARN CDR [14], the SNC guideline has both a CT and in-hospital observation management option, depending on the risk group. The rate for mandatory CT (moderate or high-risk mild TBI according to the guidelines) was only 5%, which increases to 23% if children with medium-risk mild TBI or multiple risk factors (observation or CT according to the guideline) were all to receive a CT.
No patients requiring neurosurgery would be discharged according to the SNC guideline. One patient needing neurosurgery was assigned to the 6-h inhospital observation group and another 12 patients needing neurosurgery to the in-hospital observation or CT groups. The present study did not include necessary details to examine if the SNC observation routines, mandating a CT scan when a fall in GCS or new/progressive Fig. 2 Distribution of children from the validation cohort (n = 19,007) in the different Scandinavian Neurotrauma Committee (SNC) guideline risk groups. Corresponding outcomes are provided with percentages. GCS Glasgow Come Scale, CT computed tomography, TBI traumatic brain injury, NS neurosurgery, ciTBI clinically important traumatic brain injury, CT+ brain injury on CT (see text for details). *p < 0.05, **p < 0.001 for differences of ciTBI between groups (Fisher's exact test)  [20]. Overall, this approach may be more expensive than a CT option [13], but removes the logistics and potential radiation risks associated with CT scans. The corresponding numbers for mandatory CT for the PECARN CDR are not known, though the CT rate would probably be higher due to presence of GCS 14 and altered mental status as predictors for mandatory CT. The presence of these risk factors was a major issue when the SNC workgroup were deciding on the adaptation of an external guideline (PECARN) or synthesising a new, evidence-based guideline. We chose to use the latter strategy, as the group found GCS 14 to be too unreliable as a risk factor to recommend a mandatory CT [27][28][29] and altered mental status too complicated to use effectively, with potential to lead to unacceptable increases in CT rates in Scandinavia. For this reason, allowing an element of physician judgement in the medium-risk group was chosen. Unlike other guidelines, the SNC stratifies patients into multiple risk groups for important outcomes. This allows physicians to further understand the potential impact of management in patients. Our analysis confirms the stratification, with higher risk groups showing significantly higher rates of important outcome, such as ciTBI, with gradual reduction of these rates with decreasing risk Fig. 2.
High-energy trauma mechanism is not a strict risk factor in the SNC guidelines. These patients are relatively uncommon in Scandinavia and are managed according to separate clinical trauma protocols. Most receive CT scanning and all of these children are admitted. This risk factor was also judged as complicated to use, having a specific definition and often including assessment of fall height, vehicle speed and number of stairs. In the validation cohort, 1543 patients were involved in high-energy trauma, 65 had brain injury on CT, 50 had ciTBI and 9 needed neurosurgery. All patients needing neurosurgery were identified by other predictors included in the SNC guideline. This suggests that omitting this risk factor is safe in the presence of other risk factors included in the guideline.
Children with suspicion of non-accidental injury (NAI) are always admitted to hospital in Scandinavia and generally receive diagnostic imaging. However, this is not a defined risk factor in the actual guideline, although it is clearly stated that these children should be admitted independent of TBI predictors [20].
In adult TBI management, biomarkers, specifically S100B, have been recommended in clinical guidelines as they could reduce CT rates and overall costs [30] and studies in children have shown promise [31]. Such a biomarker would be most valuable in the intermediate risk groups, such as the medium-and low-risk groups from the SNC guidelines (i.e. the groups presently managed with in-hospital observation), especially considering that today's clinical predictors seem to have reached their full potential for decision making. Indeed, the actual CT rate from the APHIRST study was only 8.9% in the comparison cohort [17], indicating that clinical guidelines may have limited effect in situations with high-baseline clinician accuracy and low CT rates [32]. However, the evidence base for S100B is too weak for a clinical recommendation in children. Other potential biomarkers have shown promise in adults [33], but studies in children are lacking.
Ultimately, the choice of a guideline will be dependent on the baseline situation and intended effect in the health care setting. Before the SNC guideline, most Swedish hospitals did not have official management pathways for paediatric head injury [34] and many used the SNC guideline from 2000 [13], intended for adults. Although the PECARN rules are based upon a rigorously powered cohort and are externally validated, Scandinavian experts were reluctant to recommend these rules for clinical practice, instead opting for a pragmatic, universal and comprehensive evidence-based option [20]. The results of this study support this approach.
The main strength of this study is the large dataset which was robustly powered and prospectively collected. Also, the dataset was adopted into the guideline by an author (SH) unconnected with the SNC group. However, a limitation is that the dataset was not designed with the CT computed tomography SNC guidelines in mind (published first after the study was commenced), but to assess the accuracy of PECARN, CHALICE and CATCH. Clinical predictors were, however, identical in most cases. The few cases where clinical predictors were approximated would likely not have affected the overall performance of the guideline. Additionally, the clinical setting was that of Australian and New Zealand emergency predominately tertiary departments, which may differ from care in the Scandinavian countries for which the SNC guideline was developed.

Conclusion
In this study, we were able to apply the clinical SNC head injury guideline to a large, previously collected data set of head injured children. The evidence-based SNC head injury guideline was externally assessed in terms of its accuracy and found to have a high sensitivity, missing very few patients with ciTBI and none needing neurosurgery. The present validation study supports the clinical use of the guideline, although national validation in Scandinavian countries may also be warranted.