Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Assessment of redundant randomized clinical trials among patients with ST segment elevation myocardial infarction

BMC Medicine volume 21, Article number: 69 (2023) Cite this article

Abstract

Background

Redundant clinical trials waste resources and unnecessarily put patients at risk for harm. The objectives of the study were to assess redundant randomized clinical trials (RCTs) conducted in mainland China or the USA among patients with ST segment elevation myocardial infarction (STEMI) and estimate the harm to patients enrolled in redundant RCTs.

Methods

We searched bibliographic databases for eligible RCTs comparing a routine therapy with a placebo or no treatment among patients with STEMI in mainland China or the United States. The routine therapy for STEMI included reperfusion (percutaneous coronary intervention or fibrinolytic therapy), P2Y12 receptor inhibitors, statins, and anticoagulants. Redundant RCTs were defined as those initiated or continued recruiting new patients 1 year after the experimental intervention was established as routine therapy in clinical practice guidelines. Cumulative meta-analyses were conducted to confirm the efficacy of these routine therapies.

The primary outcome was the number of extra major adverse cardiac events (MACEs) attributable to the deprivation of routine therapies among patients in the control groups of redundant RCTs—that is, the number of extra MACEs that could have been prevented had these patients received routine therapy.

Results

Nine hundred eighty-three eligible RCTs conducted in mainland China were identified, of which 775 (78.8%) were redundant. None of the five eligible RCTs conducted in the United States were redundant. All redundant RCTs have reiterated the benefits of routine therapies for patients with STEMI, while none were cited by the 2019 clinical practice guideline for the management of STEMI.

The 18,819 patients in the control groups of redundant RCTs experienced 3305 (95% CI: 3169–3441) extra MACEs, including 1091 (1014–1165) deaths, 576 (519–633) recurrent myocardial infarctions, 31 (19–42) revascularizations, 39 (23–54) strokes, 744 (679–810) heart failures, and 823 (754–893) patients with recurrent or exacerbated angina pectoris. Cumulative meta-analyses confirmed the efficacy of the routine therapies among patients in mainland China and supported using practice guidelines to define redundant RCTs.

Conclusions

Redundant RCTs conducted in mainland China have resulted in unnecessary MACEs among patients with STEMI. While the reasons behind redundant RCTs need to be further investigated, these results suggest potential research waste and violation of research ethics.

Peer Review reports

Background

Randomized clinical trials (RCTs) consume substantial resources and put patients at risk [1, 2]. Therefore, whether an RCT is needed deserves careful consideration. As a prerequisite, new RCTs can be conducted if the current evidence is insufficient or there is uncertainty. Similarly, new RCTs using a placebo or no treatment as control can be initiated if the efficacy of the experimental therapy remains unclear [3]. When the clinical question is settled, new RCTs waste resources and are considered redundant [4, 5]. Moreover, in redundant RCTS, patients in the control group are denied a therapy already known to be effective, raising serious concerns about research ethics [6].

Redundant RCTs have been identified as prevalent in many fields [6,7,8], although this has been less well studied in developing countries that conduct fewer RCTs. Since 2016, mainland China has been producing more scientific publications than any other country [9], but its fastest-growing output of biomedical research appears to include substantial redundancy [10, 11]. For example, it has been estimated that between 2008 and 2019, more than 2000 redundant RCTs were conducted in mainland China [12]. Those RCTs compared statins, drugs already determined to be beneficial, with placebo or no treatment among patients with coronary artery disease. Patients in the control groups were denied statins and, as a result, may have suffered over 3000 unnecessary major cardiac adverse events (MACEs), including more than 500 deaths, that could have been prevented had they received statins [12].

It remains unclear whether such redundancy is limited to statins and whether other countries conduct redundant RCTs. To partially fill these knowledge gaps, we identified and analyzed redundant RCTs assessing three routine therapies in addition to statins for ST elevation myocardial infarction (STEMI), a type of myocardial infarction characterized by symptoms of myocardial ischemia with persistent electrocardiographic ST elevation and biomarkers of myocardial necrosis [13]. We compared the results between mainland China and the United States (US), the only country that publishes a similar number of scientific publications to mainland China [14].

Methods

In this meta-research study, we identified eligible RCTs comparing four routine therapies with placebo or no treatment among patients with STEMI, determined which, if any, of these RCTs were redundant, and estimated the extra MACEs caused by the deprivation of routine therapies in redundant RCTs. This study was not subject to institutional review board approval. We followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cross-sectional studies [15]. This study was not registered.

Definition of eligible RCTs

Eligible patients

We focused on patients with STEMI. RCTs were excluded if aimed at patients with comorbidities, for example, patients with both STEMI and heart failure. We relied on the diagnosis used in each RCT, i.e., we considered an RCT as recruiting patients with STEMI if it reported recruiting patients with STEMI using its own diagnostic criteria. When the diagnosis was not clearly described in an RCT, we assumed patients to have STEMI if the reduction of elevated ST segment was an outcome in this RCT or if all patients were eligible for fibrinolytic therapy. This assumption was made because only patients with STEMI were eligible for fibrinolytic therapy [16,17,18].

Eligible therapies

We included therapies widely embraced by the clinical community as routine therapies. Specifically, therapies were considered routine if they were strongly (i.e., Recommendation Class I) and consistently recommended by clinical practice guidelines (CPGs) based on sufficient evidence (i.e., Level of Evidence A) in both mainland China and the US.

In April 2021, we screened the CPGs developed by the Chinese Society of Cardiology and the American College of Cardiology/American Heart Association. We identified four routine therapies:

  1. 1.

    Reperfusion, including primary percutaneous coronary intervention (PCI) and fibrinolytic therapy. Fibrinolytic therapy could be performed by tenecteplase, reteplase, alteplase, streptokinase, urokinase, or prourokinase. RCTs evaluating delayed PCI or fibrinolytic therapy were excluded.

  2. 2.

    P2Y12 receptor inhibitors, including clopidogrel, ticagrelor, and prasugrel. RCTs evaluating loading P2Y12 receptor inhibitors administered before PCI or fibrinolytic therapy were excluded.

  3. 3.

    Statins, including atorvastatin, fluvastatin, lovastatin, pravastatin, rosuvastatin, simvastatin, and pitavastatin. We excluded RCTs evaluating loading statin therapy administered before PCI or fibrinolytic therapy.

  4. 4.

    Anticoagulants, including unfractionated heparin, enoxaparin, fondaparinux, and bivalirudin. We only included RCTs evaluating anticoagulants administered with fibrinolytic therapy.

Eligible RCTs

We included RCTs that compared one routine therapy with a placebo or no treatment (standard therapy) for patients with STEMI. Routine therapy might be performed by different types of drugs or operations. For example, reperfusion might be performed by PCI or tenecteplase, while anticoagulation might be performed by unfractionated heparin or enoxaparin. Therefore, we excluded RCTs comparing one type of routine therapy with a placebo/no treatment among patients who all received another type of the same routine therapy. For example, we excluded RCTs comparing PCI with no treatment among patients who all received tenecteplase; we also excluded RCTs comparing enoxaparin with placebo among patients who all received unfractionated heparin.

Randomization could be performed by random number tables, computer programs, flipping a coin, etc. [19]. We also included RCTs that did not specify the randomization method. There was no restriction on the time or setting of the RCTs. We only considered RCTs in which the recruiting centers were exclusively located in mainland China or the US. If the locations of the recruiting centers were not reported, we assumed the recruiting centers of an RCT were exclusively located in mainland China or the US if all the authors were affiliated with institutes that were exclusively located in mainland China or the US.

Definition of redundant RCTs

Redundant RCTs could be defined by either CPGs or cumulative meta-analyses (CMAs), i.e., RCTs were considered redundant if conducted after the efficacy of routine therapies was confirmed by either CPGs or CMAs. We expected that CPGs would confirm the efficacy of routine therapies later than CMAs [12]. To be more conservative, we used CPGs in the primary analysis to define redundant RCTs. In the sensitivity analyses, we performed CMAs to verify the identification of redundant trials.

Entirely redundant RCTs

An RCT was entirely redundant if its recruitment started 1 year after the experimental therapy was established as routine therapy by CPGs for the first time and remained so in the following updates [12]. The 1-year “grace period” allows researchers to learn about the CPGs and terminate their RCTs.

Partially redundant RCTs

An RCT was partially redundant if it continued to recruit new patients 1 year after the experimental therapy was established as routine therapy by CPGs for the first time and remained so in the following updates.

Cutoff times

As shown in Table 1, the cutoff times defining redundant RCTs in mainland China were identified from two CPGs published in 2007 (for statins) and 2010 (for reperfusion, P2Y12 receptor inhibitors, and anticoagulants), while the two CPGs for the US were published in 2004 (for reperfusion) and 2008 (for P2Y12 receptor inhibitors, statins, and anticoagulants) [20,21,22,23].

Table 1 Cutoff times defining redundant RCTs
Full size table

Literature search

We conducted a literature search according to the requirement for systematic reviews and reported search according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension (PRISMA-S) [19, 24]. Because a proportion of RCTs conducted in mainland China were published in Chinese journals, we searched two English (PubMed and Embase) plus four Chinese bibliographic databases (the China National Knowledge Infrastructure, Wanfang Data, VIP data, and SinoMed) for eligible RCTs published until June 16, 2021, when the search was conducted [25]. We did not search other sources, such as trial registries, grey literature, references of eligible RCTs, or contact the authors for unpublished RCTs. Only RCTs published as journal articles were considered. The search strategies were developed with a literature search specialist and are listed in Additional file 1: Search Strategy.

Two authors (YJ and LJ) independently screened the title/abstract and full text of the records retrieved from bibliographic databases. Discrepancies were discussed and solved with a third author (WW). We compared similar studies with overlapping authors, facilities recruiting patients, sample size, or outcomes to identify possible duplicates [26].

Risk of bias assessment

We conducted a risk of bias assessment on MACEs in eligible RCTs according to the Risk of Bias Tool developed by Cochrane [27]. Four types of bias were evaluated: random sequence generation (selection bias), allocation concealment (selection bias), blinding of participants and researchers (performance bias), and blinding of outcome assessment (detection bias). The risk of bias assessment was performed by two authors independently (YJ and JL). Discrepancies were solved with a third author (WW).

Statistical analysis

Descriptive analysis

We reported the characteristics of eligible RCTs, including the number of eligible RCTs, the number of patients recruited, the country where they were conducted, and the result of the risk of bias assessment. We also reported the factors that might be attributed to redundant RCTs, including funding sources, approval from an ethics committee, registration, and conflicts of interest [12]. We classified the conclusions of redundant RCTs as positive (routine therapies were effective and safe) or negative (routine therapies were ineffective or unsafe) based on their conclusions.

As auxiliary analyses, we reported the percentage of redundant RCTs cited in the latest CPGs for the management of STEMI in mainland China and the US [16]. We also reported the percentage of journals publishing redundant RCTs that endorse or follow the recommendations from the International Committee of Medical Journal Editors (ICMJE), as indicated either on their website or in the list maintained by the ICMJE [28].

Primary analysis

We expected that the patients in the control group of redundant RCTs would experience more MACEs (referred to as extra MACE) than those in the routine therapy group. The primary outcome was the number of extra MACEs experienced by patients who did not receive the routine therapy in redundant RCTs (i.e., patients randomized to the control group). To accommodate the wide range of clinical events reported in individual RCTs, we defined a broad MACE to include all-cause mortality or cardiac-related mortality, recurrent myocardial infarction, stroke, heart failure, revascularization, and recurrent or exacerbated angina pectoris. We relied on the RCTs to define and report these components of MACE.

We estimated the extra MACEs by calculating the risk difference between the intervention and the control group in redundant RCTs—that is, the difference between the observed incidence of MACEs in the control group and the expected incidence of MACEs if patients in the control group had been treated as in the routine therapy group. In other words, the extra MACEs could have been prevented if patients in the control group had been given the routine therapy.

We only considered the extra MACE that occurred in the redundant phase of eligible RCTs. While all extra MACEs in entirely redundant RCTs were considered, in partially redundant RCTs, only those that occurred after the cutoff times were considered, assuming a smooth rate during the study period. We used bootstrapping to construct the 95% confidence intervals of extra MACEs (Additional file 2: Additional Method). The primary analysis was performed in SAS 9.4.

Sensitivity analyses

We performed CMAs to verify the assumption that CMAs confirmed the efficacy of routine therapies earlier than CPGs. A year was considered the cutoff time if the 95% confidence interval of the cumulative risk difference did not cross zero for the first time this year and remained so in the following years. We planned to conduct CMAs separately for mainland China and the US. However, because only five eligible RCTs were from the US, the CMAs were only performed among eligible RCTs from mainland China. CMAs were conducted using Stata 16.

Results

Characteristics of redundant RCTs

We identified 983 eligible RCTs conducted in mainland China (Additional file 3: Figure A1), of which 775 (78.8%) were redundant, including 631 (64.2%) entirely redundant RCTs and 144 (14.6%) partially redundant RCTs (Table 2). A total of 45,024 patients were recruited in eligible RCTs, 35,930 (79.8%) of which were recruited in redundant RCTs. Most redundant RCTs assessed reperfusion (366 RCTs, 47.2% of redundant RCTs), followed by P2Y12 receptor inhibitors (247, 31.9%), and statins (126, 16.3%). Similarly, redundant RCTs assessing reperfusion recruited the most patients (16,462 patients, 45.8% of all patients in redundant RCTs), followed by P2Y12 receptor inhibitors (11,520, 32.1%), and statins (6211, 17.3%). Only 122 (12.4%) eligible RCTs conducted in mainland China reported approval from an ethics committee, 23 (2.3%) reported funding sources, and 541 (55.0%) reported MACE as an outcome.

Table 2 Characteristics of eligible RCTs
Full size table

Meanwhile, we identified five eligible RCTs that were conducted in the US; none was redundant. The five RCTs recruited 284 patients in control groups before relevant cutoff times for defining redundant trials and recruited none after. Three eligible RCTs conducted in the US reported an approval from an ethics committee, three reported funding sources, and four reported MACE as an outcome.

None of the redundant RCTs reported conflicts of interest or were registered in a trial registry. Except for two published in English-language journals, 775 redundant RCTs were published in 138 Chinese-language journals. None of the 140 journals stated following the recommendations from the ICMJE, either on their website or in the list maintained by the ICMJE.

Figure 1 shows the distribution of redundant and non-redundant RCTs over time in the US and mainland China. All five RCTs from the US were published before 1995—before their first corresponding RCT in mainland China was published. Starting in 2009, the number of redundant RCTs from mainland China increased and reached a peak in 2016, subsequently decreasing in number and leveling off through 2020—a pattern paralleled by the number of control patients (Fig. 2).

Fig. 1
figure 1

Distribution of included RCTs by year of publishing. China Mainland vs. the US. The distribution of eligible RCTs by year of publishing for mainland China and the US. CPG, clinical practice guideline; RCT, randomized clinical trial

Full size image
Fig. 2
figure 2

Distribution of patients recruited in the control group by year of publishing. China Mainland vs. the US. The distribution of the patients recruited in the control group of eligible RCTs by year of publishing for mainland China and the US. CPG, clinical practice guideline; RCT, randomized clinical trial

Full size image

All 775 redundant RCTs concluded that the routine therapies were superior to the control, based on either clinical outcomes or surrogate outcomes. None was cited in the subsequent Chinese CPG for the management of STEMI in 2019.

The result of the risk of bias assessment is shown in Table 3. The risk of bias from random sequence generation was rated as low in 90 (16.6%) redundant RCTs compared with 4 (2.6%) non-redundant RCTs conducted in mainland China. However, the risk of bias from allocation concealment was rated as unclear, while the risk of bias from blinding of participants and researchers and blinding of outcome assessment was rated as high in most eligible RCTs conducted in mainland China, regardless of redundancy.

Table 3 Risk of bias assessment on MACE in eligible RCTs
Full size table

Primary outcome

The MACEs were reported as an outcome in 390 of 775 (50.3%) redundant RCTs. The 18,819 patients in the control groups of redundant RCTs experienced 3305 (95%CI: 3169 to 3441) extra MACEs, including 1091 (1014 to 1165) deaths, 576 (519 to 633) recurrent myocardial infarctions, 31 (19 to 42) revascularizations, 39 (23 to 54) strokes, 744 (679 to 810) heart failures, 823 (754 to 893) recurrent or exacerbated angina pectoris, and 4 (2 to 7) unspecified MACEs (Table 4). The redundant RCTs assessing reperfusion reported the most extra MACEs (1535, 46.4%), followed by P2Y12 receptor inhibitors (914, 27.7%), and statins (631, 19.1%). Similarly, the redundant RCTs assessing reperfusion reported the most extra deaths (617, 56.6%), followed by P2Y12 receptor inhibitors (253, 23.2%), and statins (159, 14.6%).

Table 4 Number of extra MACEs in the control groups
Full size table

Sensitivity analysis

We performed CMAs for eligible RCTs conducted in mainland China. The CMAs showed that reperfusion, P2Y12 receptor inhibitors, statins, and anticoagulants could significantly reduce the incidence of death and recurrent myocardial infarction among Chinese patients with STEMI by 1996 (Additional file 4: Figure A2), 2007 (Additional file 5: Figure A3), and 2006 (Additional file 6: Figure A4), respectively, and remained so in the following years. As expected, the times when CMAs confirmed the efficacy of reperfusion, P2Y12 receptor inhibitors, and statins were before the corresponding CPGs were published. The only exception was anticoagulants, for which the efficacy was confirmed by both CPGs and CMAs in 2010 in mainland China (Additional file 7: Figure A5) and remained so in the following years.

Discussion

Main findings

This study revealed the massive scale of redundant RCTs assessing four routine therapies among patients with STEMI conducted in China. These redundant RCTs wasted resources and only reiterated prior findings and had a limited impact on CPGs. Moreover, the conduct of these redundant RCTs led to over 3000 unnecessary MACEs that could have been prevented had those patients not been recruited.

This study illustrates the massive redundancy of RCTs in mainland China. Moreover, such redundancy was not found in the US. The temporal distribution of redundant RCTs identified in the current study was similar to our previous study on statins [12]. Although it remains unclear why the number of redundant RCTs peaked in 2016, the overall temporal trend implies that more redundant RCTs are likely to be initiated soon unless drastic measures are taken immediately by stakeholders.

Strength of the study

We applied a series of conservative methods to reduce controversies and improve the credibility of the results. First, we used CPGs to identify the cutoff times to define redundant RCTs rather than CMAs. Although both CPGs and CMAs could be used to identify the cutoff times [12, 29, 30], those identified by CMAs were earlier and more aggressive than those by CPGs [12]. This study reiterated that more redundant RCTs would have been identified if CMAs had been used to identify the cutoff times. Second, we added a 1-year grace period for researchers to learn the CPGs and terminate their RCTs, leading to fewer redundant RCTs. Third, we assessed the redundancy of eligible RCTs for mainland China and the US separately. When developing CPGs, the expert committee usually considers all available evidence, regardless of the country where they were conducted. Similarly, the RCTs in the US should be considered when assessing the redundancy of the RCTs conducted in mainland China. However, the efficacy or safety of some therapies may differ across countries due to genetic or economic factors [31,32,33]. Even though some therapies have already been confirmed as beneficial and safe in the US, bridging RCTs reaffirming their benefits and safety in the Chinese population can still be justified. Therefore, assessing the redundancy of eligible RCTs for mainland China and the US separately would rule out the potential justification of country-level diversity and result in more conservative but less controversial results.

We did not restrict the demographic characteristics of patients recruited in eligible RCTs. Therefore, redundant RCTs may be diverse in patients' demographic characteristics. However, this would not affect the validity of the results and conclusions of our study for two reasons. First, CPGs recommended routine therapies for all patients with STEMI without contradictions, regardless of demographic characteristics. Second, there were no signs of significant heterogeneity in the effect of routine therapies in a specific population. As shown in this study, all redundant RCTs have reiterated the efficacy and safety of routine therapies.

We relied on the RCTs to define and report the components of MACEs, of which the diagnosis criteria may vary across RCTs, leading to misclassifying some MACEs. However, this non-differential misclassification would underestimate the risk difference between the routine therapy group and the control group in RCTs and, thus, underestimate the extra MACEs. As a result, the validity of the results or conclusions of this study would not be affected.

Reasons behind the redundancy

Scientific publications are used to evaluate clinician performance for promotion in mainland China. The need for promotion, in some settings, has become a major incentive to conduct clinical research [34, 35]. Under this pressure, clinicians with inadequate training in clinical trials may conduct RCTs for the sole purpose of publication [34, 36]. Pharmaceutical companies could also be involved, although that is difficult to determine as none of the redundant RCTs disclosed conflicts of interest and few reported funding sources. A lack of training of junior clinicians on conducting ethical and legitimate clinical research may also play a key role [34, 36]. For example, very few redundant RCTs identified in this study cited systematic reviews or CPGs. Had the authors of redundant RCTs examined prior systematic reviews or CPGs before initiating the RCTs, they would have known the lack of justification for performing the RCTs.

Recently, the Chinese government has been trying to change the focus from quantity to quality of publications. This endeavor can potentially increase the scientific value of clinical publications, reduce waste, and save lives [37]. Although there has been dispute over whether the conduct of clinical research should be required for all clinicians in mainland China [38], offering those not capable of or not interested in conducting clinical research the option to focus on clinical practice may help reduce redundant trials.

To be conducted and published, a redundant RCT must pass supervision from several stakeholders. Therefore, the scale of redundancy we found implies oversight failures throughout the research system. First, 15.6% of redundant RCTs reported approval from an ethics committee, implying that those ethics committees failed in their responsibility to protect patients. We could not determine why approval from an ethics committee was not reported in the rest of the redundant RCTs. Second, 2.4% of redundant RCTs reported funding sources, implying that those funders failed to scrutinize the scientific merits of the redundant RCTs. Sometimes, the funders may find it challenging to assess each research proposal’s scientific merit thoroughly. However, they could request applicants provide a summary of prior evidence, e.g., a systematic review, as a prerequisite for approval, which could help reduce research waste [39]. Third, trial registries can display the completed and ongoing clinical trials to the research community and thus reduce waste [40, 41]. Unfortunately, none of the redundant RCTs were registered, leaving the research community unaware of those RCTs until their publication. This finding was consistent with our prior study [12]. Fourth, none of the 140 journals publishing the redundant RCTs stated they followed the ICMJE. Some of these journals might pursue profit rather than scientific merits, including so-called predatory journals, which usually fail to perform peer-review [42].

Limitations

There are several limitations of this study. First, we may have underestimated the number of redundant RCTs and the unnecessary MACEs because we only relied on journal articles. More than half of redundant RCTs did not report MACEs, while some redundant RCTs may never be published or reported only in grey literature. In addition, we could not determine the relevancy of many journal articles because of missing key information.

Second, we may have overestimated the number of redundant RCTs and the unnecessary MACEs because we took the journal articles at their face value. Evidence suggests that the methods and results reported in journal articles may deviate from the actual conduct [43,44,45]. For example, a cohort study might be reported as an RCT. In addition, the publications of some redundant RCTs shared similar wording and structure, which may indicate that these trials were partially or entirely fabricated by either individual researchers or essay mills, the companies producing faked publications for profit. However, we did not assess this suspicion further. Although the pre-clinical research field has already been polluted by partially or entirely faked publications [46], it remains unclear whether such misconduct has seeped into clinical research in mainland China.

When cohort studies are misreported as RCTs, the unnecessary MACEs could still be attributed to the deprivation of routine therapies because the patients in the control groups were reportedly comparable to those in the routine therapy group and eligible for routine therapies without contraindications. If the journal articles were partially or entirely fabricated, the number of redundant RCTs and unnecessary MACEs could have been overestimated.

Third, we only searched PubMed and Embase for eligible RCTs published in English journals. We did not search the Cochrane Central Register of Controlled Trials (CENTRAL) because CENTRAL did not provide the function to search the records by authors’ affiliations, while we used keywords such as “China” or “United States” to restrict the authors’ affiliations when searching PubMed and Embase. A recent study showed that the combination of PubMed and Embase could cover almost 90% of studies included in the Cochrane reviews [47]. Therefore, we were confident that most eligible RCTs were identified; the few we might have missed would not significantly impact the results and conclusions of this study.

Fourth, we relied on the diagnosis of STEMI in the eligible RCTs. The diagnostic criteria adopted in redundant RCTs were generally similar to those defined in CPGs, while the details may vary. The deviation from CPGs may lead to the inclusion of patients in redundant RCTs who should not be considered STEMI. However, this would not affect the validity of the results and conclusions from our study. The similarity of diagnosis criteria between eligible RCTs and CPGs implies that a proportion of patients in redundant RCTs still had “true” STEMI. Therefore, the authors should not conduct placebo or standard care controlled RCTs and deprive those patients of routine therapies. If researchers planned to assess the benefits of routine therapies for the patients who had STEMI by their own criteria but not by the CPGs criteria, those patients who had STEMI by the CPGs criteria should be excluded from the RCTs. Unfortunately, none of the redundant RCTs stated that.

Fifth, the quality of eligible RCTs conducted in mainland China was generally low, including those included in the CMAs. Therefore, the result of CMAs based on these high- or unclear-risk RCTs may not be valid. However, this would not impact the validity of the definition of redundant RCTs. Suppose redundant RCTs, which were published later, were initiated to address the low quality of non-redundant RCTs. In that case, the quality of redundant RCTs should be significantly better than non-redundant RCTs. Unfortunately, the risk of bias assessment indicated that the quality of redundant RCTs was also low. Therefore, the low quality of non-redundant RCTs could not justify the initiation of redundant RCTs.

Future directions

Although our study has shown the large scale of redundant RCTs in mainland China, the reasons behind this phenomenon remain unclear. We recommend in-depth surveys or qualitative studies of the stakeholders, including patients participating in the redundant RCTs, the authors of redundant RCTs, the ethics committees, the funders, and the journal editors. If cohort studies were reported as RCTs, we need to understand why the patients in the control groups did not receive the routine therapies, e.g., patients’ refusal due to personal belief or financial difficulty, clinician ignorance or negligence, or a shortage of medication supplies. There is a need to develop methods to identify errors in journal articles, including intentional errors and even if articles were partially or entirely fabricated [48]. Finally, the reporting quality of clinical trials in mainland China has to be improved to facilitate future research and to facilitate the use of the trial results in CPGs.

The Chinese government has been making substantial efforts to reduce research redundancy. For example, the Drug Clinical Trial Registry Platform was established in 2012, where all commercial trials supervised by the National Medical Products Administration are required to be registered [49]. The National Health Commission has been updating the ethical standards for investigator-initiated clinical trials [50]. The Chinese government could ensure that the current laws and regulations are followed while expanding their efforts to involve additional stakeholders. For example, trial registration should be mandated for all clinical trials, as being required by the updated Declaration of Helsinki [51]; the public funders should focus more on the scientific merits of proposals; the training on clinicians for conducting ethical and high-quality research should be reinforced; and the evaluation on the performance of clinicians should be balanced between research and clinical practice.

Our study only assessed redundant RCTs in cardiology in mainland China and the US. It is unclear whether the results apply to other clinical fields or countries, including the scope of redundancy, the harm to patients, or the reasons for redundancy. Future studies are needed to explore potential redundant RCTs in other clinical fields or countries on a case-by-case basis. Although CPGs were used to define redundant RCTs in this study, we did not imply that all recommendations in CPGs, regardless of the evidence base, should be followed without further examination. Researchers are encouraged to assess the validity of recommendations in CPGs without adequate evidence.

Conclusions

A total of 775 redundant RCTs recruiting patients with STEMI have been conducted in mainland China; no redundant trials were identified from the US. These redundant RCTs may have led to over 3000 extra MACEs, including 1091 deaths, that could have been prevented had these RCTs were not conducted. While the reasons behind redundant RCTs need to be further investigated, these results suggest potential research waste and a violation of research ethics.

Availability of data and materials

The data are available by emailing Yuanxi Jia at yx.jia@siat.ac.cn.

Abbreviations

CENTRAL:

Cochrane Central Register of Controlled Trials

CMA:

Cumulative meta-analysis

CPG:

Clinical practice guideline

ICMJE:

International Committee of Medical Journal Editors

MACE:

Major adverse cardiac events

PCI:

Percutaneous coronary intervention

PRISMA-S:

Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension

RCT:

Randomized clinical trial

STEMI:

ST segment elevation myocardial infarction

STROBE:

Strengthening the Reporting of Observational Studies in Epidemiology

US:

United States

References

  1. Speich B, Schur N, Gryaznov D, von Niederhäusern B, Hemkens LG, Schandelmaier S, et al. Resource use, costs, and approval times for planning and preparing a randomized clinical trial before and after the implementation of the new Swiss human research legislation. PLoS One. 2019;14(1):e0210669.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Johnston SC, Rootenberg JD, Katrak S, Smith WS, Elkins JS. Effect of a US National Institutes of Health programme of clinical trials on public health and costs. Lancet. 2006;367(9519):1319–27.

    Article  PubMed  Google Scholar 

  3. Millum J, Grady C. The ethics of placebo-controlled trials: methodological justifications. Contemp Clin Trials. 2013;36(2):510–4.

    Article  PubMed  Google Scholar 

  4. Freedman B. Clinical equipoise and the ethics of clinical research. N Engl J Med. 1987;317:141–5.

    Article  CAS  PubMed  Google Scholar 

  5. Borgerson K. Redundant, secretive, and isolated: when are clinical trials scientifically valid? Kennedy Inst Ethics J. 2014;24(4):385–411.

    Article  PubMed  Google Scholar 

  6. Bolland MJ, Avenell A, Grey A. Assessment of research waste part 1: an exemplar from examining study design, surrogate and clinical endpoints in studies of calcium intake and vitamin D supplementation. BMC Med Res Methodol. 2018;18(1):1–10.

    Google Scholar 

  7. Bolland MJ, Grey A, Avenell A. Assessment of research waste part 2: wrong study populations-an exemplar of baseline vitamin D status of participants in trials of vitamin D supplementation. BMC Med Res Methodol. 2018;18(1):1–14.

    Google Scholar 

  8. Vergara-Merino L, Verdejo C, Franco JVA, Liquitay CE, Urrútia G, Klabunde R, et al. Registered trials address questions already answered with high-certainty evidence: A sample of current redundant research. J Clin Epidemiol. 2021;134:89–94.

    Article  PubMed  Google Scholar 

  9. Tollefson J. China declared world's largest producer of scientific articles. Nature. 2018;553(7686):390-1.

  10. Huang F. Quality deficit belies the hype. Nature. 2018;564(7735):S70.

    Article  CAS  PubMed  Google Scholar 

  11. Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485–514.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Jia Y, Wen J, Qureshi R, Ehrhardt S, Celentano DD, Wei X, et al. Effect of redundant clinical trials from mainland China evaluating statins in patients with coronary artery disease: cross sectional study. BMJ. 2021;372:n48.

    Article  PubMed  PubMed Central  Google Scholar 

  13. O'gara PT, Kushner FG, Ascheim DD, Casey DE, Chung MK, De Lemos JA, et al. 2013 ACCF/AHA guideline for the management of ST-elevation myocardial infarction: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2013;61(4):e78–e140.

    Article  PubMed  Google Scholar 

  14. Tollefson J. China declared world’s largest producer of scientific articles. Nature. 2018;553(7686):390–1.

    Article  CAS  PubMed  Google Scholar 

  15. Vandenbroucke JP, Von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007;4(10):e297.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Chinese Society of Cardiology. 2019 Chinese Society of Cardiology (CSC) guidelines for the diagnosis and management of patients with ST-segment elevation myocardial infarction. Chin J Cardiol. 2019;47(10):18.

    Google Scholar 

  17. Smith SC, Benjamin EJ, Bonow RO, Braun LT, Creager MA, Franklin BA, et al. AHA/ACCF secondary prevention and risk reduction therapy for patients with coronary and other atherosclerotic vascular disease: 2011 update: a guideline from the American Heart Association and American College of Cardiology Foundation endorsed by the World Heart Federation and the Preventive Cardiovascular Nurses Association. J Am Coll Cardiol. 2011;58(23):2432–46.

    Article  PubMed  Google Scholar 

  18. Chinese Society of Cardiology. Guidelines for the diagnosis and management of patients with non-ST segment elevation myocardial infarction. Chin J Cardiol. 2017;45(5):18.

    Google Scholar 

  19. Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions: John Wiley & Sons; 2019.

    Book  Google Scholar 

  20. Chinese Society of Cardiology. Guideline for diagnosis and treatment of patients with ST-elevation myocardial infarction. Chin J Cardiol. 2010;38(8):675–90.

    Google Scholar 

  21. Chinese Society of Cardiology. Guideline for diagnosis and treatment of patients with unstable angina and non- ST-segment elevation myocardial infarction (no abstract). Chin J Cardiol. 2007;35(4):295–304.

    Google Scholar 

  22. Members WC, Antman EM, Anbe DT, Armstrong PW, Bates ER, Green LA, et al. ACC/AHA guidelines for the management of patients with ST-elevation myocardial infarction—executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 1999 Guidelines for the Management of Patients With Acute Myocardial Infarction). J Am Coll Cardiol. 2004;44(3):671–719.

    Article  Google Scholar 

  23. Guidelines ACoCAHATFoP, Society DiCWtCC, Physicians EbtAAoF, Evidence WGtRN, Update the ACC/AHA Guidelines for the Management of Patients With ST-Elevation Myocardial Infarction WoBotWC, Antman EM, et al. 2007 focused update of the ACC/AHA 2004 guidelines for the management of patients with ST-elevation myocardial infarction. J Am Coll Cardiol. 2008;51(2):210–47.

    Article  Google Scholar 

  24. Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, et al. PRISMA-S: an extension to the PRISMA statement for reporting literature searches in systematic reviews. Syst Rev. 2021;10(1):1–19.

    Article  Google Scholar 

  25. Jia Y, Huang D, Wen J, Wang Y, Rosman L, Chen Q, et al. Assessment of language and indexing biases among Chinese-sponsored randomized clinical trials. JAMA Netw Open. 2020;3(5):e205894.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Jia Y, Huang D, Wen J, Qureshi R, Wang Y, Rosman L, et al. Assessment of duplicate publication of chinese-sponsored randomized clinical trials. JAMA Netw Open. 2020;3(12):e2027104.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

    Article  PubMed  PubMed Central  Google Scholar 

  28. International Committee of Medical Journal Editors. Journals stating that they follow the ICMJE Recommendations 2021. Available from: http://www.icmje.org/journals-following-the-icmje-recommendations/#C.

  29. Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992;327(4):248–54.

    Article  CAS  PubMed  Google Scholar 

  30. Clarke M, Brice A, Chalmers I. Accumulating research: a systematic account of how cumulative meta-analyses would have provided knowledge, improved health, reduced harm and saved resources. PLoS One. 2014;9(7):e102670.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Hackler E III, Lew J, Gore MO, Ayers CR, Atzler D, Khera A, et al. Racial differences in cardiovascular biomarkers in the general population. J Am Heart Assoc. 2019;8(18):e012729.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Johnson JA. Ethnic differences in cardiovascular drug response: potential contribution of pharmacogenetics. Circulation. 2008;118(13):1383–93.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Exner DV, Dries DL, Domanski MJ, Cohn JN. Lesser response to angiotensin-converting–enzyme inhibitor therapy in black as compared with white patients with left ventricular dysfunction. N Engl J Med. 2001;344(18):1351–7.

    Article  CAS  PubMed  Google Scholar 

  34. Guanghui Jin YZ, Shao S, Qin X. Survery on the current research activities in general practitioners[全科医生科研工作现状调查研究]. Contin. Med Educ. 2017;31(12):2.

    Google Scholar 

  35. Liao J. Investigation and analysis on the current situation of clinicians’ scientific research cognition in a hospital of class A grade three. Modern Hosp Manage. 2020;18(1):5.

    Google Scholar 

  36. Renjie Lu HY, Ruan L, Chen Y, Zhang Y. Investigation on status of scientific research among young doctors in a infectious disease hospital of Changzhou city. Med Soc (Berkeley). 2018;31(11):3.

    Google Scholar 

  37. Zhang L, Sivertsen G. The new research assessment reform in China and its implementation. Sch Asses Rep. 2020;2(1):3.

  38. Zhang Z, Winston GP, Zhao H-T, Oei EH, Ai Q, Loffroy R, et al. Focus on China: should clinicians engage in research? and lessons from other countries. Quant Imaging Med Surg. 2014;4(5):413.

    PubMed  PubMed Central  Google Scholar 

  39. Clarke M, Hopewell S, Chalmers I. Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting. Lancet. 2010;376(9734):20–1.

    Article  PubMed  Google Scholar 

  40. Krleža-Jerić K, Chan A-W, Dickersin K, Sim I, Grimshaw J, Gluud C. Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1). BMJ. 2005;330(7497):956–8.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Dickersin K, Rennie D. Registering clinical trials. JAMA. 2003;290(4):516–23.

    Article  PubMed  Google Scholar 

  42. Clark J, Smith R. Firm action needed on predatory journals. BMJ. 2015;350:h210.

  43. Wu T, Li Y, Bian Z, Liu G, Moher D. Randomized trials published in some Chinese journals: how many are randomized? Trials. 2009;10(1):1–8.

    Article  Google Scholar 

  44. Tong Z, Li F, Ogawa Y, Watanabe N, Furukawa TA. Quality of randomized controlled trials of new generation antidepressants and antipsychotics identified in the China National Knowledge Infrastructure (CNKI): a literature and telephone interview study. BMC Med Res Methodol. 2018;18(1):1–11.

    Article  Google Scholar 

  45. Jia Y, Huang D, Wen J, Liang J, Qureshi R, Wang Y, et al. Association between switching of primary outcomes and reported trial findings among randomized drug trials from China. J Clin Epidemiol. 2021;132:10–7.

    Article  PubMed  Google Scholar 

  46. Else H, Van Noorden R. The fight against fake-paper factories that churn out sham science. Nature. 2021;591(7851):516–9.

    Article  CAS  PubMed  Google Scholar 

  47. Frandsen TF, Eriksen MB, Hammer DMG, Christensen JB, Wallin JA. Using Embase as a supplement to PubMed in Cochrane reviews differed across fields. J Clin Epidemiol. 2021;133:24–31.

    Article  PubMed  Google Scholar 

  48. Parker L, Boughton S, Lawrence R, Bero L. ‘Somebody might actually make up data’: an empirical qualitative study on how to screen for fake research. Available at SSRN: https://ssrn.com/abstract=4071671 or https://doi.org/10.2139/ssrn.4071671.

  49. Center for Drug Reevaluation of the China National Medical Products Administration. History of Drug Clinical Trial Registry Platform 2020. Available from: http://www.chinadrugtrials.org.cn/snipet/434.html.

  50. National Health Commission of the People's Republic of China. Announcement on Public Consultation on the Measures for the Ethical Review of Life Sciences and Medical Research Involving Human Beings (Draft for Solicitation of Comments) 2021. Available from: http://www.nhc.gov.cn/cms-search/xxgk/getManuscriptXxgk.htm?id=beb66b1525e64472b1a9b8921ed1aedf.

  51. World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191–4.

Download references

Acknowledgements

We thank Dr. Lance Rodewald at the National Immunization Program, Chinese Center for Disease Control and Prevention, for editing and improving the manuscript. We also thank Mr. Qingkun Chen from the Institute of Information at the Peking Union Medical College for help the literature search.

Funding

This study was not funded.

Author information

Authors and Affiliations

  1. Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

    Yuanxi Jia

  2. Blood Station, Tongzhou, Beijing, China

    Jun Liang

  3. Department of Cardiology and Institute of Vascular Medicine, Peking University Third Hospital, Peking University, 49 Huayuan N Rd, Haidian District, Beijing, 100191, China

    Wenyao Wang

  4. Department of Cardiology, Virginia Commonwealth University, Richmond, USA

    Xin Wei

  5. Department of Pediatrics, School of Medicine, Johns Hopkins University, Baltimore, USA

    Shaoming Xiao

  6. Department of Medicine, School of Medicine, Johns Hopkins University, Baltimore, USA

    Karen A. Robinson

Authors
  1. Yuanxi Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenyao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shaoming Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Karen A. Robinson
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: YJ, KAR; methodology: YJ, JL, KAR; investigation: YJ, JL, WW, WX, SX; supervision: KAR; writing—original draft: YJ; writing—review and editing: YJ, JL, WW, WX, SX, KAR. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Wenyao Wang.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Search Strategy.

Additional file 2: Additional Method.

The steps to calculate the confidence interval of extra MACEs are presented.

Additional file 3: Figure A1.

The steps to select Eligible RCTs from bibliographic databases are presented.

Additional file 4: Figure A2.

Cumulative Meta-Analysis for Reperfusion. This figure shows the result of a cumulative meta-analysis for RCTs assessing reperfusion conducted in mainland China. Only the first 50 RCTs were analyzed due to the limit of Stata.

Additional file 5: Figure A3.

Cumulative Meta-Analysis for P2Y12 Receptor Inhibitors. This figure shows the result of a cumulative meta-analysis for RCTs assessing P2Y12 receptor inhibitors conducted in mainland China. Only the first 50 RCTs were analyzed due to the limit of Stata.

Additional file 6: Figure A4.

Cumulative Meta-Analysis for Statins. This figure shows the result of a cumulative meta-analysis for RCTs assessing statins conducted in mainland China. Only the first 50 RCTs were analyzed due to the limit of Stata.

Additional file 7: Figure A5.

Cumulative Meta-Analysis for Anticoagulants. This figure shows the result of a cumulative meta-analysis for RCTs assessing anticoagulants conducted in mainland China.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jia, Y., Liang, J., Wang, W. et al. Assessment of redundant randomized clinical trials among patients with ST segment elevation myocardial infarction. BMC Med 21, 69 (2023). https://doi.org/10.1186/s12916-023-02749-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12916-023-02749-2

Keywords

BMC Medicine

ISSN: 1741-7015

Contact us