Skip to main content

Table 3 Meta-analyses of the proportions of associations for each criterion and level of evidence (random effects)

From: Validity of observational evidence on putative risk and protective factors: appraisal of 3744 meta-analyses on 57 topics

 

n/N associations (crude proportion)

Proportions

[95% CI] from meta-analysis of 57 topicsa

I2

Range of proportions across topics

Level of evidence:

 Convincing

99/3744 (2.6%)

1.3% [1.0%; 2.2%]

73.9%

0–16.7%

 Highly suggestive

253/3744 (6.7%)

4.6% [2.9%; 6.6%]

85.7%

0–33.3%

 Suggestive

440/3744 (11.8%)

11.0% [8.5%; 13.8%]

83.9%

0–50%

 Weak

1497/3744 (40.0%)

39.1% [34.8%; 43.5%]

86.2%

0–71.4%

 Non-significant

1455/3744 (38.9%)

34.7% [29.2%; 40.3%]

90.8%

0–80.7%

Criteria:

 Statistical significance

  P < 10-6

762/2289 (33.3%)

29.0% [24.9%; 33.3%]

74.8%

0–66.7%

  P < 0.001

1377/2289 (60.2%)

58.6% [54.1%; 63.0%]

73.3%

0–100%

 Cases > 1000 (or > 20,000 participants for continuous outcomes)

1182/2107 (56.1%)

65.3% [56.9%; 73.2%]

94.9%

1.7–100%

 Largest study with P < 0.05

1343/1781 (75.4%)

74.9% [71.2%; 78.4%]

63.6%

28.6–100%

 95% prediction interval that excluded the null

642/2136 (30.1%)

30.3% [26.5%; 34.2%]

71.0%

9.0–100%

 Absence of large heterogeneity (I2<50%)

1050/2277 (46.1%)

46.6% [41.8%; 51.3%]

79.5%

0–88.2%

 No evidence of small study effects (P > 0.10)

1628/2164 (75.2%)

75.3% [72.2%; 78.3%]

63.2%

40–100%

 No evidence of excess significance (P > 0.10)

1599/2052 (77.9%)

77.7% [72.6%; 82.5%]

83.0%

33.3–100%

  1. aMeta-analyses for the individual criteria excluded associations with missing data. Meta-analyses for the levels of evidence were conducted across all 3744 associations regardless of their statistical significance status. The meta-analyses for the individual criteria were conducted across the 2289 statistically significant associations. Out of 2289, statistically significant associations, 182 associations did not report on the number of cases,508 on whether the largest study had P < 0.05, 153 on the 95% prediction interval, 12 on the I2 for heterogeneity, 125 on the small study effect test, and 237 on the excess of significance bias. Data on all 7 criteria were available for 1457 statistically significant meta-analyses