Skip to main content

Table 1 Common issues in pre-specifying statistical analysis approaches in clinical trial protocols

From: How to design a pre-specified statistical analysis approach to limit p-hacking in clinical trials: the Pre-SPEC framework

  

Estimated prevalence

Issue

Problems associated with issue

Aspect

Prevalencea

Omitting an aspect of the analysis approach

Investigators could run multiple analyses, and selectively report the most favourable

Analysis population:

Analysis model:

Covariates:

Missing data:

27-47%

11-20%

27%

66-77%

Insufficient detail around an aspect of the analysis approach

Investigators could run multiple analyses, and selectively report the most favourable

Analysis population:

Analysis model:

Covariates:

Missing data:

64%

42%

23%

17%b

Analysis approach allows some aspects of the final analysis to be subjectively chosen based on trial data

Investigators could run multiple analyses, and selectively report the most favourable

Analysis model:

Covariates:

19%

8%

Multiple analysis approaches specified, without one being identified as the primary

Investigators could selectively report the most favourable result, or to elevate its importance compared to less favourable results.

Analysis population:

Analysis model:

Covariates:

Missing data:

11%

11%

9%

2%

  1. aBased on references [5] and [2]; one study evaluated protocols and published results for 70 randomised trials approved by the ethics committees for Copenhagen and Frederiksberg, Denmark in 1994-5; the other study evaluated 100 protocols of randomised trials indexed in PubMed November 2016.
  2. b15/99 protocols gave insufficient detail around how they planned to implement multiple imputation, 2/99 protocols but gave insufficient detail around their planned inverse probability weighting procedure