Skip to main content

Table 2 Characteristics of included studies classified according to reproduction attempts

From: Reproducibility of clinical research in critical care: a scoping review

 

Practices WITHOUT a reproduction attempt (n = 92)

Practices with CONSISTENT EFFECT between original study and reproduction attempt (n = 28)

Practices with INCONSISTENT EFFECT between original study and reproduction attempt (n = 35)

Characteristic, n (%)a

Original study (n = 93)b

Original study (n = 30)b

Reproduction attempt (n = 39)b

Original study (n = 37)b

Reproduction attempt (n = 63)b

 Primary electronic search

93 (100)

20 (67)

25 (64)

29 (78)

51 (81)

 Secondary electronic search

0 (0)

10 (33)

14 (36)

8 (22)

12 (19)

Continent of origin

 North America

44 (47)

13 (43)

12 (31)

18 (49)

20 (32)

 Europe

42 (45)

15 (50)

21 (54)

15 (41)

37 (59)

 Australasia

7 (8)

1 (3)

4 (10)

2 (5)

4 (6)

 Other

0 (0)

1 (3)

2 (5)

2 (5)

2 (3)

Year of publicatione,f

 Before 1980

3 (3)

0 (0)

0 (0)

1 (3)

0 (0)

 1980–1989

7 (8)

2 (7)

1 (3)

4 (11)

3 (5)

 1990–1999

25 (27)

14 (47)

6 (15)

13 (35)

12 (19)

 2000–2009

20 (22)

11 (37)

21 (54)

17 (46)

21 (33)

 2010 or later

38 (41)

3 (10)

11 (28)

2 (5)

27 (43)

Participating center typef

 University affiliated

38 (41)

17 (57)

19 (49)

29 (78)

23 (37)

 Mixed university affiliated and non-affiliated

11 (12)

5 (17)

11 (28)

1 (3)

13 (21)

 Unclear

44 (47)

8 (27)

9 (23)

7 (19)

27 (43)

No. of centres, mean (95% CI)e,f

7.2 (5.1–9.9)

4.0 (2.2–7.0)

9.1 (5.4–15.3)

2.9 (1.8–4.6)

16.2 (11.1–23.4)

 1e,f

25 (26)

11 (37)

9 (23)

17 (46)

7 (11)

 2–4e,f

13 (14)

8 (27)

4 (10)

8 (22)

5 (8)

 5–9e,f

12 (13)

3 (10)

5 (13)

6 (16)

5 (8)

  ≥ 10e,f

44 (47)

8 (27)

21 (54)

6 (16)

46 (73)

No. of participants, mean (95% CI)e,f

362.6 (266.3–493.6)

155.9 (105.5–230.4)

344.8 (223.9–531.2)

146.7 (96.5–222.8)

548.5 (408.8–735.7)

 < 100d,e,f,g

17 (18)

9 (30)

8 (21)

16 (43)

3 (5)

 100–499d,e,f,g

40 (43)

15 (50)

12 (31)

16 (43)

29 (46)

 500–999d,e,f,g

16 (17)

6 (20)

13 (33)

1 (3)

13 (21)

  ≥ 1000d,e,f,g

20 (22)

0 (0)

6 (15)

4 (11)

18 (29)

Target condition

     

 General critical illness

10 (11)

2 (7)

2 (5)

5 (14)

12 (19)

 Respiratory

24 (26)

13 (43)

19 (49)

13 (35)

23 (37)

  ARDS

4 (4)

5 (17)

8 (21)

5 (14)

7 (11)

  Mechanical ventilation (excluding ARDS)

11 (12)

3 (10)

4 (10)

4 (11)

8 (13)

  Respiratory failure (without ventilation)

9 (10)

5 (17)

7 (18)

4 (11)

8 (13)

 Sepsis

13 (14)

6 (20)

7 (18)

7 (19)

14 (22)

 Nosocomial complications

11 (12)

5 (17)

3 (8)

3 (8)

3 (5)

 Neurological

12 (13)

2 (7)

1 (3)

5 (14)

8 (13)

 Acute kidney injury

6 (6)

1 (3)

5 (13)

3 (8)

3 (5)

 General resuscitation

9 (10)

0 (0)

1 (3)

1 (3)

0 (0)

 Trauma

3 (3)

0 (0)

0 (0)

0 (0)

0 (0)

 Other

5 (5)

1 (3)

1 (3)

0 (0)

0 (0)

Type of intervention

 Drug

48 (52)

14 (47)

16 (41)

18 (49)

26 (41)

 Device/procedure

20 (22)

13 (43)

20 (51)

14 (38)

23 (37)

 Protocol

11 (12)

2 (7)

2 (5)

4 (11)

13 (21)

 Other

14 (15)

1 (3)

1 (3)

1 (3)

1 (1)

Intervention effect estimatef

 Lack of efficacy

51 (55)

16 (53)

20 (51)

10 (27)

38 (60)

 Efficacy

31 (33)

11 (37)

15 (38)

23 (62)

16 (25)

 Harm

11 (12)

3 (10)

4 (10)

4 (11)

9 (14)

Funding

     

 Non-commercial

46 (49)

12 (40)

23 (59)

9 (24)

29 (46)

 Commercial

17 (18)

5 (17)

4 (10)

11 (30)

15 (24)

 Both commercial and non-commercial

14 (15)

2 (7)

4 (10)

7 (19)

8 (13)

 Not reported

15 (16)

10 (33)

8 (21)

10 (27)

11 (17)

 None

1 (1)

1 (3)

0 (0)

0 (0)

0 (0)

Study stopped early

     

 Futility

2 (2)

0 (0)

3 (8)

0 (0)

5 (8)

 Benefit

1 (1)

0 (0)

2 (5)

4 (11)

2 (3)

 Harm

2 (2)

1 (3)

1 (3)

1 (3)

4 (6)

 Recruitment/lack of funding

1 (1)

1 (3)

0 (0)

0 (0)

3 (5)

  1. ARDS acute respiratory distress syndrome, ICU intensive care unit, IQR interquartile range, RCT randomized controlled trial
  2. aContinuous data are reported as geometric mean (95% confidence interval) and nominal data as number (%)
  3. bThe 275 included articles described 158 unique practices that were examined in 283 studies. A ‘study’ is a comparison of an intervention with control. The number of studies exceeds the number of included articles because of 8 articles that simultaneously reported 2 separate studies [34,35,36,37,38,39,40,41]; 21 studies were excluded from the data in this table since the reproduction attempt was not yet completed for 13 studies and due to the following 8 practices for which representative studies did not consistently meet our criteria for results reproducibility: chlorhexidine skin antiseptic for central venous catheter insertion, naloxone for patients with sepsis, stress ulcer prophylaxis for prevention of gastrointestinal bleeding, systemic steroids in ARDS, pulmonary surfactant in ARDS, reduction of ventilator-associated pneumonia by various methods, trophic enteral nutrition, and daily interruption of sedatives. Data refer to 262 studies unless otherwise stated
  4. cPrimary electronic search: New England Journal of Medicine, The Lancet, JAMA. Secondary electronic search: Annals of Internal Medicine, BMJ, American Journal of Respiratory and Critical Care Medicine, Chest, Critical Care Medicine, Intensive Care Medicine, Critical Care, clinicaltrials.gov, controlled-trials.com, bibliographies of included studies
  5. dP < 0.05 for comparison of reproduction attempts between practices with consistent and inconsistent effect estimates
  6. eP < 0.05 for comparison between original evaluation and reproduction attempt among practices demonstrating consistent effects
  7. fP < 0.05 for comparison between original evaluation and reproduction attempt among practices demonstrating inconsistent effects
  8. gP < 0.05 for comparison of original evaluations between practices with consistent and inconsistent effect estimates