Skip to main content

Table 3 Tasks sorted by participants in the Q-sort survey. Data are (overall rank) and mean ± SD rank of participants

From: The most important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks most often requested by journal editors

  Total Clinician Methodologist Both Other
(N = 203) (N = 93) (N = 72) (N = 17) (N = 21)
To evaluate the risk of bias of the trial (1) 2.1 ± 2.0 (5) 1.3 ± 2.1 (1) 2.8 ± 1.8 (1) 3.6 ± 0.8 (1) 2.5 ± 1.6
To determine if the manuscript conclusion is consistent with the results (2) 1.9 ± 1.9 (1) 1.7 ± 2.0 (3) 2.1 ± 1.8 (2) 2.4 ± 2.1 (3) 1.9 ± 1.9
To evaluate the adequacy of statistical analyses (3) 1.8 ± 1.7 (6) 1.2 ± 1.6 (2) 2.6 ± 1.7 (3) 2.2 ± 1.6 (5) 1.6 ± 1.5
To evaluate if the control group is appropriate (4) 1.4 ± 1.8 (8) 1.1 ± 1.7 (4) 1.9 ± 1.8 (8) 1.2 ± 1.4 (6) 1.4 ± 2.1
To check if all outcomes are adequately reported (5) 1.4 ± 2.0 (9) 1.1 ± 2.1 (5) 1.7 ± 1.9 (5) 1.5 ± 1.6 (4) 1.9 ± 1.9
To evaluate the relevance of the primary outcome(s) (6) 1.3 ± 2.1 (4) 1.3 ± 2.1 (6) 1.5 ± 2.1 (14) 0.6 ± 2.6 (8) 1.1 ± 1.7
To search for any attempt to distort the presentation or interpretation of results (7) 1.2 ± 2.0 (11) 0.9 ± 2.0 (7) 1.3 ± 1.8 (4) 1.9 ± 2.5 (2) 2.2 ± 2.1
To evaluate the reliability and validity of the outcome measures (8) 1.1 ± 2.0 (3) 1.4 ± 2.2 (11) 0.9 ± 1.9 (12) 0.7 ± 2.1 (9) 1.0 ± 1.9
To evaluate the importance of the study (9) 1.1 ± 2.4 (2) 1.7 ± 2.4 (14) 0.6 ± 2.5 (17) -0.1 ± 2.0 (13) 0.7 ± 2.0
To evaluate if the abstract conclusion is consistent with the results (10) 1.0 ± 1.9 (10) 1.0 ± 2.1 (9) 1.2 ± 1.7 (9) 1.2 ± 1.9 (20) 0.0 ± 1.7
To evaluate if the discussion is consistent with the results (11) 1.0 ± 1.8 (7) 1.2 ± 1.7 (13) 0.7 ± 1.7 (13) 0.7 ± 1.9 (7) 1.2 ± 2.1
To check if all adverse events are adequately reported (12) 0.9 ± 1.9 (14) 0.7 ± 2.0 (8) 1.2 ± 1.7 (6) 1.4 ± 1.9 (10) 0.9 ± 2.1
To check if the intervention is described with enough details to allow replication (13) 0.8 ± 1.8 (16) 0.6 ± 1.8 (10) 1.1 ± 1.7 (10) 1.0 ± 1.8 (18) 0.2 ± 1.9
To check that limitations are adequately reported (14) 0.7 ± 1.7 (17) 0.4 ± 1.8 (12) 0.8 ± 1.6 (7) 1.3 ± 1.7 (11) 0.9 ± 1.6
To evaluate the adequacy of the selection of participants and clinical setting (15) 0.5 ± 1.9 (13) 0.7 ± 1.9 (15) 0.5 ± 1.8 (15) 0.4 ± 1.7 (17) 0.2 ± 2.0
To search for any inconsistencies or errors in the manuscript (16) 0.3 ± 2.2 (21) 0.0 ± 2.1 (17) 0.3 ± 2.2 (11) 1.0 ± 2.4 (14) 0.6 ± 2.0
To evaluate the novelty of the study (17) 0.2 ± 2.4 (12) 0.8 ± 2.5 (20) –0.1 ± 2.0 (21) –0.3 ± 2.1 (24) –0.5 ± 2.7
To check the sample size calculation (18) 0.2 ± 2.1 (18) 0.4 ± 1.9 (18) –0.1 ± 2.4 (24) –0.5 ± 2.0 (15) 0.5 ± 2.1
To check if the authors reported all important outcomes and adverse events in the abstract (19) 0.1 ± 2.1 (22) 0.0 ± 2.3 (16) 0.4 ± 1.9 (20) –0.2 ± 1.8 (23) –0.3 ± 2.3
To discuss the results in relation to other studies (20) 0.0 ± 1.9 (19) 0.3 ± 2.0 (19) –0.1 ± 1.6 (30) –1.0 ± 2.1 (19) 0.1 ± 2.0
To evaluate if the manuscript can be suspected of fraud (21) –0.1 ± 2.6 (15) 0.7 ± 2.7 (30) –1.1 ± 2.4 (29) –0.9 ± 1.6 (12) 0.9 ± 2.3
To provide recommendations on publication (22) –0.2 ± 2.5 (20) 0.3 ± 2.8 (23) –0.5 ± 2.2 (22) –0.4 ± 2.3 (26) –0.7 ± 2.3
To check if all figures and tables are consistent with the text (23) –0.2 ± 1.7 (23) –0.1 ± 1.7 (21) –0.4 ± 1.6 (19) –0.2 ± 2.0 (21) –0.1 ± 1.6
To evaluate clarity of presentation (24) –0.5 ± 1.9 (26) –0.5 ± 2.0 (22) –0.5 ± 1.9 (28) –0.8 ± 1.6 (27) –0.7 ± 1.7
To check if the study reported ethics review board approval (25) –0.5 ± 2.2 (25) –0.3 ± 2.3 (27) –0.9 ± 2.1 (23) –0.4 ± 2.1 (22) –0.3 ± 2.3
To search for plagiarism or imitation in the paper (26) –0.7 ± 2.3 (24) –0.1 ± 2.4 (33) –1.4 ± 2.2 (32) –1.4 ± 2.0 (16) 0.2 ± 1.9
To compare information recorded in the trial protocol when provided by the authors and reported in the manuscript (27) –0.7 ± 2.3 (27) –0.8 ± 2.2 (24) –0.6 ± 2.5 (18) –0.2 ± 2.2 (30) –1.0 ± 2.1
To check if the items requested by the CONSORT Statement are adequately reported by authors (28) –0.9 ± 2.3 (29) –1.1 ± 2.2 (26) –0.8 ± 2.2 (16) 0.1 ± 2.6 (25) –0.5 ± 2.6
To check if the authors referenced all important studies (29) –1.1 ± 1.7 (28) –1.1 ± 1.8 (25) –0.8 ± 1.7 (33) –1.6 ± 1.6 (33) –1.5 ± 1.2
To evaluate whether figures and tables can be understood without having to refer the text (30) –1.1 ± 1.9 (30) –1.1 ± 1.9 (31) –1.2 ± 1.8 (26) –0.7 ± 2.0 (29) –1.0 ± 2.2
To check if items requested by the CONSORT extensions are adequately reported when appropriate (31) –1.2 ± 1.9 (32) –1.5 ± 2.0 (29) –1.1 ± 1.8 (25) –0.7 ± 1.6 (28) –0.8 ± 2.0
To compare information recorded on a clinical trials register such as ClinicalTrials.gov and reported in the manuscript (32) –1.3 ± 2.1 (31) –1.5 ± 2.1 (28) –1.0 ± 2.3 (31) –1.4 ± 1.7 (32) –1.5 ± 2.3
To read the journals’ recommendations to reviewers (33) –1.4 ± 2.0 (33) –1.7 ± 1.9 (32) –1.2 ± 1.7 (27) –0.8 ± 2.5 (31) –1.1 ± 2.6
To evaluate all appendices when available (34) –2.3 ± 1.6 (34) –2.2 ± 1.8 (34) –2.5 ± 1.4 (34) –1.8 ± 1.8 (34) –2.8 ± 1.1
To evaluate the adequacy of the language (35) –2.9 ± 1.7 (35) –2.7 ± 1.9 (35) –3.0 ± 1.5 (35) –3.1 ± 1.3 (35) –3.4 ± 1.4
To evaluate if authors respect the requested format for references (36) –4.0 ± 1.4 (36) –3.8 ± 1.6 (36) –4.0 ± 1.3 (36) –4.4 ± 1.2 (36) –4.0 ± 1.4