Skip to main content

Table 4 Results—factors associated with overall preference for each study design feature: parameter estimates [and 95% confidence intervals]. For each independent variable, parameter estimates represent mean difference in overall preference associated with each category of independent variable as compared with the reference (after adjusting for all other variables in the table and after taking into account the reading order of the 2 vignettes of the pair)

From: Designs of trials assessing interventions to improve the peer review process: a vignette-based survey

  Training peer reviewers
(24 vignettes, 276 pairs)
Adding an expert to the peer review process
(10 vignettes, 90 pairs*)
Use of reporting guidelines checklist
(13 vignettes, 156 pairs*)
Results free peer review
(24 vignettes, 276 pairs)
Using incentives
(13 vignettes, 156 pairs*)
Post-publication peer review
(10 vignettes, 90 pairs*)
Estimate [95% CI] Estimate [95% CI] Estimate [95% CI] Estimate [95% CI] Estimate [95% CI] Estimate [95% CI]
Study type
 RCT with randomization of manuscripts 0.92
[-0.50 ; 2.41]
2.03
[0.51 ; 3.49]
2.69
[1.39 ; 3.95]
2.53
[1.27 ; 3.76]
1.00
[-0.21 ; 2.16]
2.55
[1.13 ; 4.09]
 RCT with randomization of peer reviewers 1.45
[0.14 ; 2.78]
N/A 1.99
[0.69 ; 3.37]
2.24
[0.98 ; 3.50]
2.25
[0.94 ; 3.49]
N/A
 Cluster RCT with randomization of journals 0.30
[-1.12 ; 1.63]
0.76
[-0.90 ; 2.43]
0.34
[-1.25 ; 1.93]
0.63
[-0.56 ; 1.88]
-0.16
[-1.49 ; 1.18]
1.73
[0.13 ; 3.51]
 Interrupted time series analysis -0.10
[-1.48 ; 1.38]
-0.19
[-1.74; 1.39]
0.10
[-1.21 ; 1.44]
0.07
[-1.28 ; 1.40]
0.73
[-0.51 ; 2.02]
1.58
[0.13; 3.15]
 Pairwise comparison 0.83
[-0.49 ; 2.18]
N/A N/A 1.61
[0.35 ; 2.86]
N/A N/A
 Stepped wedge cluster RCT with randomization of journals*** 0.00
[-]
0.00
[-]
0.00
[-]
0.00
[-]
0.00
[-]
0.00
[-]
Setting
 Single biomedical journal -1.02
[-1.82 ; -0.27]
-2.62
[-4.23 ; -0.80]
-2.50
[-3.51 ; 1.50]
-0.20
[-1.03 ; 0.62]
-1.51
[-2.59 ; -0.43]
-3.13
[-4.09 ; -1.53]
 Several biomedical journals from a single publisher -0.21
[-0.84 ; 0.44]
-1.10
[-2.32 ; 0.06]
-0.12
[-1.06 ; 0.80]
0.18
[-0.52 ; 0.83]
-0.01
[-0.82 ; 0.83]
-0.92
[-1.95 ; 0.22]
 Several biomedical journals from different publishers*** 0.00
[-]
0.00
[-]
0.00
[-]
0.00
[-]
0.00
[-]
0.00
[-]
Type of manuscript
 Actual manuscripts submitted to the journal(s) 1.04
[0.37 ; 1.79]
N/A N/A 0.57
[-0.15 ; 1.26]
N/A N/A
 One fabricated manuscript*** 0.00
[-]
N/A N/A 0.00
[-]
N/A N/A
  1. N/A, not applicable
  2. Results in bold indicate the most preferred study design features for each intervention
  3. *Indicates the pairs of vignettes for these interventions were assessed twice each
  4. ***Indicates reference category for each independent variable