Skip to main content

Table 4 Level of evidence in various studies

From: Non-genetic risk and protective factors and biomarkers for neurological disorders: a meta-umbrella systematic review of umbrella reviews

Level

I / Convincing

II / Highly suggestive / Probable

III / Suggestive / Possible

IV / Weak / Limited-contrasting

Grosso, 2017 [76]

High: concordance between meta-analyses of RCTs and meta-analyses of observational studies; low: meta-analyses of RCTs with results contrary to those from meta-analyses of observational studies

High: meta-analyses of prospective studies with no heterogeneity, no potential confounding factors identified, and agreement of results over time and among meta-analyses, including studies with different designs; medium: meta-analyses of prospective studies with no heterogeneity and no potential confounding factors identified; low: meta-analyses of prospective and case-control studies with no heterogeneity and no potential confounding factors identified

High: meta-analyses of prospective studies lacking information on heterogeneity and potential confounding factors; medium: meta-analyses of prospective and case-control studies lacking information on heterogeneity and potential confounding factors; low: meta-analyses of case-control studies or meta-analyses of any other study design with significant heterogeneity (I2 > 50%) and potential confounding factors

Limited studies included in meta-analyses (n ≤ 3) or evident contrasting results from meta-analyses with the same level of evidence

Veronese, 2018 [77]

Veronese, 2019 [83]

Li, 2017 [86]

Statistical significance with p < 10− 6, more than 1000 cases (or > 20,000 participants for continuous outcomes), the largest component study reported statistically significant effect (p < 0.05); 95% PI excluded the null; no large heterogeneity (I2 < 50%), no evidence of small-study effects (p > 0.10) and excess significance bias (p > 0.10)

Statistical significance with p < 10− 6, more than 1000 cases (or > 20,000 participants for continuous outcomes), the largest component study reported statistically significant effect (p < 0.05)

Statistical significance with p < 10− 3, more than 1000 cases (or > 20,000 participants for continuous outcomes)

The remaining statistically significant associations with p < 0.05.

Dinu, 2018 [81]

Significance threshold reached at p ≤ 0.001 for both random and fixed effects calculation; > 1000 cases (or > 5000 total participants if the metric was continuous); not large heterogeneity between studies (I2 < 50%); 95% PI excluding the null value; no evidence of small-study effects (if it could be tested)

Significance threshold reached at p ≤ 0.001 for both random and fixed effects calculation; > 1000 cases (or > 5000 total participants if the metric was continuous); not considerable heterogeneity between studies (I2 = 50–75%)

Significance threshold reached at p ≤ 0.001 for random effect calculation; 500–1000 cases (or 2500–5000 total participants if the metric was continuous)

Significance threshold reached at p ≤ 0.05 for random effects calculation

Theodoratou, 2014 [52]

Evidence existed from both observational studies and RCTs, and association/effect was of the same direction, statistically significant at p ≤ 0.001, and free from bias

Evidence existed from both observational studies and RCTs, and association/effect was of the same direction and statistically significant at p ≤ 0.001, but excess significance could not be tested; or evidence existed from RCTs and effect was statistically significant at p ≤ 0.001 and with no contrary results from observational data (that is, systematic reviews, if any exist, are also definitive or suggestive and meta-analyses of observational studies, if any exist, are in the same direction)

Suggestive: Evidence from RCTs with an effect at 0.001 ≤ p ≤ 0.05 and with no contrary results from observational data (same as above); or evidence from meta-analyses of observational studies showing an association at p ≤ 0.001, with no contrary results from randomized data (that is, meta-analysis of RCTs, if present, have effects in the same direction) and, if it could be tested, no evidence of small-study effects (p ≥ 0.10), not very large heterogeneity (I2 ≤ 75%), no evidence for excess significance, based on cumulative evidence of more than 500 disease events (or more than 5000 total participants if type of metric was continuous)

[Substantial effect unlikely]: Evidence from observational studies or RCTs enough to conclude that a substantial effect is unlikely based on the magnitude and the significance level

Belbasis, 2016 [78]

Belbasis, 2015 [18, 51]

More than 1000 cases, significant summary associations (p < 0.001) per random effects calculations, no evidence of small-study effects, no evidence for excess significance bias, PI not including the null, and not large heterogeneity (I2 ≤ 50%)effects and excess significance

[No such category exists in these studies]

Nominally significant summary associations (p < 0.05) per random effects calculations, no evidence of small-study effects, no evidence for excess significance bias, and not large heterogeneity (I2 < 50%)

All other risk factors with nominally significant summary associations (p < 0.05);

Bellou, 2017 [49]

Bellou, 2016 [50]

The associations that fulfilled all the following criteria: statistical significance according to random effects model at p < 10− 6; based on more than 1000 cases; without large between-study heterogeneity (I2 < 50%); 95% PI excluding the null value; and no evidence of small-study effects and excess significance

Associations with > 1000 cases, p < 10− 6, and largest study presenting a statistically significant effect (with 95% CI excluding the null value)

The associations supported by > 1000 cases and a significant effect at p < 10− 3)

All other risk factors with nominally significant summary associations (p < 0.05)

Poole, 2017 [79]

The classification was based on AMSTAR (A Measurement tool to Assess Systematic Reviews), as following: Q1: A-priori design; Q2: Duplicate study selection and data extraction; Q3: Search comprehensiveness; Q4: Inclusion of gray literature; Q5: Included and excluded studies provided; Q6: Characteristics of the included studies provided; Q7: Scientific quality of the primary studies assessed and documented; Q8: Scientific quality of included studies used appropriately in formulating conclusions; Q9: Appropriateness of methods used to combine studies’ findings; Q10: Likelihood of publication bias was assessed; Q11: Conflict of interest-potential sources of support were clearly acknowledged in both the systematic review and the included studies.

McRae, 2017 [85]

Galbete, 2018 [84]

Posadzki, 2018 [73]

Not available levels of evidence (*)

  1. Abbreviations: RCT randomized controlled trials, PI prediction interval, 95% CI 95% confidence interval
  2. *Non-pragmatic approach was applied in this meta-umbrella review