Skip to main content

Advertisement

Registration of published randomized trials: a systematic review and meta-analysis

BMC Medicinevolume 16, Article number: 173 (2018) | Download Citation

Article metrics

Abstract

Background

Prospective trial registration is a powerful tool to prevent reporting bias. We aimed to determine the extent to which published randomized controlled trials (RCTs) were registered and registered prospectively.

Methods

We searched MEDLINE and EMBASE from January 2005 to October 2017; we also screened all articles cited by or citing included and excluded studies, and the reference lists of related reviews. We included studies that examined published RCTs and evaluated their registration status, regardless of medical specialty or language. We excluded studies that assessed RCT registration status only through mention of registration in the published RCT, without searching registries or contacting the trial investigators. Two independent reviewers blinded to the other’s work performed the selection. Following PRISMA guidelines, two investigators independently extracted data, with discrepancies resolved by consensus. We calculated pooled proportions and 95% confidence intervals using random-effects models.

Results

We analyzed 40 studies examining 8773 RCTs across a wide range of clinical specialties. The pooled proportion of registered RCTs was 53% (95% confidence interval 44% to 58%), with considerable between-study heterogeneity. A subset of 24 studies reported data on prospective registration across 5529 RCTs. The pooled proportion of prospectively registered RCTs was 20% (95% confidence interval 15% to 25%). Subgroup analyses showed that registration was higher for industry-supported and larger RCTs. A meta-regression analysis across 19 studies (5144 RCTs) showed that the proportion of registered trials significantly increased over time, with a mean proportion increase of 27%, from 25 to 52%, between 2005 and 2015.

Conclusions

The prevalence of trial registration has increased over time, but only one in five published RCTs is prospectively registered, undermining the validity and integrity of biomedical research.

Open Peer Review reports

Background

Prospective clinical trial registration is at the foundation of research transparency [1, 2]. By documenting the existence of clinical trials and providing a summary of protocol details before patients are enrolled and trial results become known, registries can prevent unnecessary duplication of trials, facilitate the identification of research gaps, and support coordination of study efforts for a disease [3]. As registration allows public scrutiny of the availability of trial results, it also provides means to identify and monitor biased reporting of trials. Several cases have highlighted how selective reporting can lead to considerable harm to patients. Rofecoxib, a nonsteroidal anti-inflammatory drug, was found to be associated with increased risk of myocardial infarction as early as 2000, but the primary trial publication selectively omitted findings on cardiovascular safety, and rofecoxib was not withdrawn from the market until several years later, in 2004 [4, 5]. Incomplete trial reporting can also lead to considerable waste of resources, as has been seen in the case of oseltamivir [6] or gabapentin [7, 8] with billions of dollars spent despite poor evidence of efficacy. Prospective trial registration—if universally implemented—can serve as a powerful tool to detect and prevent this type of publication bias and selective outcome reporting [3, 9,10,11].

The International Committee of Medical Journal Editors (ICMJE) announced in 2004 a requirement for prospective registration of clinical trials as a pre-requisite for consideration for publication in its member journals, beginning the following year [2]. In the USA, the Food and Drug Administration Amendments Act (FDAAA) has mandated prospective trial registration with ClinicalTrials.gov since 2007 for drugs, biologics, and devices subject to Food and Drug Administration (FDA) regulation. The requirements for registration and result posting have recently been expanded with the FDAAA Final Rule and a similar policy by the National Institutes of Health [12, 13]. Other countries and funders have implemented similar policies [14, 15].

The clinical trial research community has widely adopted trial registration, and trial registries have been leveraged to monitor research activity and integrity [16,17,18]. For example, trial registration data have been used to show that the results of registered trials are frequently not disseminated, either through reporting in biomedical journals or posting on ClinicalTrials.gov [19, 20]. However, several studies in specific medical specialties have suggested that many published trials are not registered or not registered prospectively, raising concerns that long-standing efforts have not succeeded in achieving universal trial registration [21,22,23]. Our objective in this systematic review was to determine the extent to which published randomized controlled trials (RCTs) were registered and registered prospectively.

Methods

Search and study selection

Studies were eligible if they were based on a sample of RCTs identified in published reports in medical journals and evaluated their registration status. All eligible studies across medical specialties were included. Studies not limited to RCTs were not eligible, unless they provided relevant data on registration of the subgroup of RCTs. Studies that determined trial registration only through mention of trial registration in the published article (i.e., the article included the trial registration number in the abstract or main text), but did not further search for information on registration status with either a search for trial records in registries or by contacting the trial investigators to inquire about registration status, were excluded. Lastly, studies that covered a majority of trials published prior to 2005 (i.e., the middle of the range of years covered was prior to 2005) were excluded.

We did not register a protocol for the review. We searched MEDLINE via PubMed and EMBASE without language restriction for studies published between January 1, 2005, and October 31, 2017. The search strategy was (trial[tiab] OR trials[tiab]) AND (registration[ti] OR registered[ti] OR unregistered[ti]) in MEDLINE and (trial:ti OR trial:ab OR trials:ti OR trials:ab) AND (registration:ti OR registered:ti OR unregistered:ti) in EMBASE. Additionally, we screened all articles that were cited by or that cited any of the included studies and studies excluded after full-text screening, and we screened the reference lists of related reviews [24,25,26] for additional eligible studies.

Two investigators screened the titles and abstracts of all records, independently and in duplicate, to identify potentially eligible studies for further assessment. Discrepancies were discussed to reach consensus. All authors then independently assessed each remaining full-text article for inclusion. We again reviewed all discrepancies, and the final list of included studies was determined by consensus among all authors.

To eliminate overlapping samples of RCTs, we compared the medical specialties, journals searched for published RCTs, and time periods covered by the studies. In cases of complete overlap (i.e., when a study sample was included in a more recent, larger study), we discarded the smaller study encompassed in the larger one. When RCTs were identified based on a specific clinical topic (e.g., RCTs of cognitive behavioral therapy and new-generation antidepressants), we considered that the study was unlikely to overlap with another study based on a search of journals of a relevant medical specialty (e.g., journals in psychology), after consideration of years and journals searched. We could not exclude the possibility that some studies that searched for RCTs across multiple specialties or in general medicine journals would not overlap with other studies. To address this, we discarded studies for which we could not completely ensure that there was no overlap from the primary analysis but included the totality of the available data in a secondary analysis.

Data extraction

Two investigators independently extracted data from each included study using a standardized data collection form. Discrepancies were resolved by consensus. For each included study, we extracted the medical specialty, the publication years of included RCTs, the number of journals searched for RCTs, the list of journals (when provided), the number of identified RCTs, and the number of registered RCTs. We also assessed how trial registration was assessed (i.e., through the reporting of a trial registration number in the article, by searching for trial records in trial registries [27], and/or by contacting corresponding authors). We noted which registries were searched and assessed whether studies had included trials that started enrolment before 2005, as the ICMJE policy was implemented in September 2005. Moreover, we noted if each study assessed if the trial registration was prospective. In that case, we also extracted the number of trials registered prospectively. According to the ICMJE, trials must register at or before the onset of patient enrollment as a condition of consideration for publication. According to FDAAA, Applicable Clinical Trials must be registered no later than 21 days after enrollment of the first participant. We considered trials to be registered prospectively, as defined by the authors of the review.

Lastly, we also extracted the number of registered trials in certain subgroups, when this information was available: in trials that started enrolment after 2005, according to publication year and trial size, and in industry-supported trials. We defined industry support as direct or indirect financial support by a company that produces drugs or medical devices.

Data synthesis

For each included study, we calculated the proportion of registered RCTs with the 95% Clopper-Pearson exact confidence interval. We assessed the heterogeneity across studies through a visual examination of a forest plot and heterogeneity statistics (Cochran’s chi-square test and between-study variance τ2). We estimated pooled proportions by using arcsine transformations and a beta-binomial random-effects model with Anscombe continuity correction [28, 29]. We examined potential small-study effects by using a funnel plot showing the relationship between the log odds of being registered and the associated standard error. Similar methods were used for the synthesis of prospective registration data. For the latter analysis, we first calculated the proportion of RCTs registered prospectively out of the total number of identified RCTs. We also calculated the proportion of RCTs registered prospectively among registered RCTs.

To examine the effect of time on registration prevalence, we conducted three analyses. We synthesized data for RCTs that started enrolment exclusively after 2005. Because the RCT start dates were frequently unclear, we also performed an analysis limited to RCTs published in 2010 or later in order to capture RCTs that were likely to have all been initiated after the implementation of the ICMJE registration policy. The year 2010 was chosen based on data indicating that the time from start of study enrollment to publication is approximately 5 years [30, 31]. Finally, in order to detect changes in registration over the study period, we examined the proportion of registered trials by publication year, within studies and across studies, by using meta-regression models. We fitted the models on the log odds scale and back transformed the fitted lines to produce a plot showing the proportion of registered trials against publication year. We also analyzed the proportion of prospectively registered trials by publication year through a meta-regression.

To further explore sources of heterogeneity in registration prevalence, we performed a subgroup analysis in trials (fully or partially) supported with funding from industry. We also examined the prevalence of registration according to the sample size of RCTs. Because different thresholds were used across studies, we reported the extracted data without meta-analysis. Finally, we examined the subset of studies that identified published RCTs from a sample of high-impact factor journals. Analyses were performed using R v3.4.1 (R Development Core Team, Vienna, Austria) with the metafor package for meta-analysis. The data and R code are provided in Additional files 1 and 2.

Results

Characteristics of included studies

We identified 40 eligible studies reported in 43 articles [21, 22, 32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72]. Figure 1 shows the selection process. We screened 2180 records and 91 full-text articles. The most common reason for excluding articles was that the assessment of trial registration was based solely on mention of a trial registration number in the article (Additional file 3: Table S1). In addition, we excluded two articles [73, 74] because the data were completely included in two other, larger studies [34, 35]. Finally, for one study [37], we excluded data for a subset of RCTs which were also covered in another study [35].

Fig. 1
figure1

Flow diagram of study selection

Full size image

Table 1 shows the characteristics of included studies. The 40 studies covered a wide range of clinical specialties. To identify RCTs, 17 (43%) studies searched a sample of high-impact factor journals, while 10 (25%) studies performed a systematic review of RCTs on a specific clinical topic. Among 27 studies that identified RCTs based on publication in a specific sample of journals, the median number of included journals was 5 (Q1–Q3 4–10). Each study examined a median of 187 RCTs (Q1–Q3 103–301). Studies examined RCTs published over a median of 3 years (Q1–Q3 1–5 years), with a median starting year of 2009. To assess the registration status of published RCTs, all studies examined if a trial registration number was reported in the published article, except four studies. All studies except one searched trial registries, 28 (70%) searched ClinicalTrials.gov, and 29 (73%) searched the World Health Organization International Clinical Trials Registry Platform (WHO ICTRP). Finally, 19 (48%) studies contacted trial investigators to inquire about registration status. In all, 18 (45%) studies used all three methods (reporting of registration number, search in registries, and contact of authors).

Table 1 Characteristics of included studies
Full size table

Prevalence of registration

We included 31 non-overlapping studies in the main analysis. The studies examined a total of 6788 RCTs, among which 3267 (48%) RCTs had been registered. The proportion of registered trials varied considerably across studies, ranging from 21 to 100% (Q statistic 1126, df = 30, p < 0.001, between-study variance 0.17). In a random-effects meta-analysis, the pooled proportion of registered RCTs was 51% (95% confidence interval 44 to 58%) (Fig. 2). In a secondary analysis, we included an additional nine studies examining 1985 RCTs, which potentially overlapped in part with the primary sample. Among these, 924 (47%) RCTs were registered. When combining all 40 studies, totaling 8773 RCTs, the pooled proportion of registered RCTs was 53% (95% confidence 46 to 59%) with considerable between-study heterogeneity. A funnel plot did not show evidence of small-study effects (Additional file 3: Figure S1).

Fig. 2
figure2

Combined estimates of the prevalence of trial registration

Full size image

Among the 31 non-overlapping studies, 19 also reported the number of trials that were registered prospectively. The studies included a total of 4272 RCTs, among which 676 (16%) RCTs were registered prospectively. There was a considerable heterogeneity across studies, with the proportion of prospectively registered RCTs ranging from 4 to 90% (Q statistic 380, df = 18, p < 0.001, between-study variance 0.09). The pooled proportion of prospectively registered RCTs was 21% (95% confidence interval 15 to 27%) (Fig. 3). Of the additional nine potentially overlapping studies included in the secondary analysis, five examined prospective registration across 1257 RCTs. When combining all 24 studies totaling 5529 RCTs, the pooled proportion of prospectively registered RCTs was 20% (95% confidence interval 15 to 25%). In addition, across these 24 studies, 1734 (67%) RCTs among 2588 registered RCTs were registered retrospectively, for a pooled proportion of 65% (95% confidence interval 59 to 71%).

Fig. 3
figure3

Combined estimates of the prevalence of prospective trial registration

Full size image

Prevalence of registration over time in study subgroups

Four studies [32, 35, 65, 66] included only studies that started to enroll participants exclusively after 2005, and another four studies [38, 47, 51, 59] included sub-analyses based on studies enrolling only after 2005. Across these eight studies totaling 938 RCTs, 622 (66%) were registered, and the pooled proportion was 65% (95% confidence interval 50 to 78%) (Additional file 3: Figure S2).

Seven studies reported data on registration according to RCT publication year (Fig. 4). Separate meta-regression models showed that the proportion of registered trials increased over time in five of these studies. Moreover, 12 studies examined RCTs published in a single year (n = 1 for 2007 [21], n = 3 for 2009 [39, 51, 61], n = 2 for 2010 [62, 70], n = 2 for 2012 [42, 45]; n = 4 for 2013 [34, 37, 64, 68]) (Additional file 3: Table S2). When combining all 19 studies, totaling 5144 RCTs, a meta-regression model showed that the proportion of registered trials increased significantly over time (p = 0.03), with a mean absolute proportion increase of 27% between 2005 and 2015, from 25 to 52%. In addition, 7 of the 12 studies that examined RCTs published in a single year reported data on prospective registration (Additional file 3: Table S2). A meta-regression model suggested that the proportion of prospectively registered trials increased as well, from 3% in 2009 to 21% in 2013 (18% increase, p = 0.04) (Additional file 3: Figure S3). Finally, in an analysis limited to studies published in 2010 or after, 26 studies reported data on 5401 RCTs. Of these, 2550 (47%) were registered, for a pooled proportion of 54% (95% confidence interval 47 to 60%).

Fig. 4
figure4

Meta-regression analysis of the prevalence of trial registration in relation to publication year. Each circle represents one study, and the size of each circle represents the weight given to the study in meta-regression. Separate meta-regression models were fitted in 7 studies that reported trial registration data by publication year. The black dashed line corresponds to an overall meta-regression model across these 7 studies with 11 studies that examined RCTs published in a single year. It showed that the proportion of registered trials increased over time, from 23% in 2005 to 52% in 2015 (29% increase, p = 0.03)

Full size image

Prevalence of registration based on certain trial features

Nine studies examined registration of published RCTs according to industry funding. Across the nine studies, 778 of 2306 RCTs (34%) were supported fully or partially by industry sources. Of the 778 RCTs, 475 (61%) were registered. The pooled proportion was 59% (95% confidence interval 47 to 71%), as compared to 43% (95% confidence interval 30 to 58%) among trials not supported by industry. Five studies examined the prevalence of registration according to the trial sample size (Table 2). In all studies, there was evidence of higher registration prevalence among larger RCTs. Finally, 17 studies identified published RCTs from high-impact factor journals only. Out of 3383 RCTs, 1724 (51%) were registered. The pooled proportion was 55% (95% confidence interval 44 to 66%) as compared to 50% (95% confidence interval 42 to 59%) in the other 23 studies (5390 RCTs).

Table 2 Prevalence of registration among published randomized trials according to trial size
Full size table

Discussion

In this systematic review, we found that, among published RCTs, the proportion of registered and prospectively registered RCTs has increased over time but lack of registration and retrospective registration are still common. In analyses of more than 8000 RCTs published in medical journals, half of the RCTs published in recent years had not been registered, and 4 in 5 published RCTs had not been registered prospectively. Registration prevalence was higher in trials supported by industry funding, in larger trials, and in trials published in high-impact factor journals. While sharing of individual participant data has recently garnered much attention, our findings highlight the need for renewed efforts to address the first step on the continuum of research transparency and make prospective trial registration a top priority. Without prospective registration, our ability to monitor and resolve issues in trial reporting is substantially diminished.

Our synthesis shows that publication of unregistered trials and of trials registered retrospectively in medical journals persists. Many journals do not endorse the ICMJE registration policy and continue to support the publication of unregistered trials [75, 76]. In a mixed-methods analysis, editors and publishers reported several reasons for why journals do not reject unregistered or retrospectively registered articles, including concerns about losing submissions or preventing publication of studies from developing countries. Conducting trials without making all the results publicly available is unethical [77, 78]. Publishing the results of unregistered trials may be considered an ethical imperative as it does ensure that the research community has access to the results of these trials. But the large prevalence of unregistered trials published in medical journals raises concerns about persistent lack of transparency, underreporting or misreporting of trials, and biases in the resulting scientific literature. It directly undermines the first key objective of registries, which is to form a public “denominator” of all initiated trials, so that trials left unpublished can be identified and the available evidence interpreted in the context of unreported trials [79, 80].

Moreover, permitting publication of retrospectively registered trials defeats the second key objective of trial registration, which is to provide timestamped amendments to trial protocols. If a trial is registered after the enrollment of the first participants, it is no longer possible to compare reported results to the original trial record in order to identify selective reporting of outcomes and analyses [81, 82]. Among published RCTs that were registered, we found that 65% had been registered retrospectively. Zarin et al. [18] found a lower proportion; among 49 751 RCTs registered in ClinicalTrials.gov between 2012 and 2014, 33% had been registered more than 3 months after the trial start. Possible reasons for the difference is that our results concern only published RCTs; in contrast, most registered trials on ClinicalTrials.gov do not report results in a timely fashion [19]. In addition, we examined studies that assessed prospective registration according to the ICMJE or FDAAA definitions, which stipulates registration prior to the onset of patient enrollment or no later than 21 days after enrollment of the first participant. A higher proportion of trials may have been classified as prospectively registered had the cutoff been 3 months. Another reason for the difference is that we included trials registered in other registries and that practices with regard to prospective registration might be different among trials registered in these. Lastly, among 123 trials rejected by the BMJ between June 2013 and June 2017 because they did not comply with ICMJE trial registration requirements, 89% were retrospectively registered and 7% were unregistered [83].

Our findings have implications for systematic reviewers. Roberts et al. have suggested that systematic reviews include only prospectively registered trials, under the premise that such trials are the only ones not affected by reporting bias. Registered and unregistered trials have been found to differ in their risk of bias in studies examining 326 RCTs from Latin America and the Caribbean and 693 RCTs of fertility treatments [44, 62]. Other investigations have examined the impact of registration status on positive study findings and have not found differences between registered and non-registered trials [38, 42, 58]. We do not endorse restricting a systematic review to only registered trials but, given the large number of unregistered trials in the current medical literature and the potential difference from registered trials, systematic review authors should conduct subgroup analyses in cases where both registered and unregistered trials contribute to a meta-analysis. Such analyses should ideally distinguish between unregistered trials, retrospectively registered trials, prospectively registered trials with potential outcome reporting bias, and prospectively registered trials with no outcome reporting bias.

Our findings also have implications for a range of stakeholder groups focused on improving trial registration. Many actions have already been implemented by journal editors, regulatory agencies, and funding organizations to tackle the lack of prospective registration [18]. Because of the inherent lag between registration and publication, we could see substantial changes in upcoming years in response to the Final Rule and the new National Institutes of Health policy. However, existing laws and policies may not be sufficient and novel interventions may be required to increase trial registration. Many organizations in the USA do not have policies, staff, or other resources needed to ensure their trials are registered and reported in a timely fashion [84]. Twenty stakeholders have recently affirmed that prospective registration is of critical importance and that they will implement policies with monitoring systems to improve registration and reporting of results. In a recent commentary, Loder suggested treating unregistered or retrospectively registered trials as medical “never events.” Such events should trigger drastic responses, similar to specified events in clinical medicine. For example, Dr. Loder argues that journal editors and peer reviewers should verify that trial registration occurred before the trial enrolment began and, according to the ICMJE policy, reject trials registered retrospectively. If not published in medical journals, trial results could still be posted online. Most importantly, we believe that multiple entities, including funding agencies, ethics committees, and academic institutions should continue to enforce standards of universal trial registration [85,86,87,88]. For example, these stakeholders could take prospective registration into account when considering full grant payments or academic promotions.

Our systematic review has limitations. First, some included studies examined RCTs that started prior to 2005 when registration requirements were implemented. However, we conducted analyses limited to RCTs started after 2005 and to RCTs published after 2010 that suggest that the low prevalence of registration among published RCTs has persisted among recent trials. Second, we could not rule out the possibility that some study samples overlapped with others among the 40 included studies. However, our primary analysis was restricted to 31 non-overlapping studies. The secondary analysis, which included all 40 studies, showed the same average prevalence of registration as in the primary analysis. Third, we were not able to fully explore sources of variability in the prevalence of registration, though we found that trial registration varied substantially across clinical fields and journals. Moreover, registration prevalence was higher among trials supported by the industry, larger trials, and trials in high-impact factor journals. Data were not available on trial location, and compliance with trial registration is likely to vary across countries. Viergever and Li [14] have shown that trends in registration on WHO ICTRP did not take place equally in all parts of the world. Fourth, our results apply to RCTs, while registration requirements apply broadly to all types of clinical trials. We therefore cannot ascertain whether the prevalence of registration we report here would be the same across all clinical study designs. Fifth, the included studies used different methods to ascertain whether published RCTs were registered. Some included studies might have missed registered RCTs and thus possibly underestimate the proportion of registered RCTs. Conversely, we excluded studies that assessed trial registration based solely on the mention of a trial registration number in the article, because such studies would underestimate the proportion of registered RCTs. However, trial registration is useful if end users can identify trial records. In this regard, the proportion of registered RCTs we found might be larger than the proportion of “useful” registrations. Sixth, data on the proportion of prospectively registered RCTs according to publication year were limited. Finally, in our systematic review, we have not assessed the quality of the registration of outcomes. The lack of a detailed specification of outcomes may also introduce reporting biases [89].

Conclusions

Non-registration and retrospective registration of clinical trials remain common, undermining the validity and integrity of biomedical research. Given long-standing policies mandating registration, enforcing prospective registration will likely require novel interventions and greater endorsement by a range of stakeholders in the research community, including investigators, funding entities, ethical oversight bodies, and journal editors. Universal, prospective trial registration should be a top priority in the endeavors to improve research transparency and ensure rigorous, high-quality evidence is available to inform patient care.

Abbreviations

ANZCTR:

Australian New Zealand Clinical Trials Registry

CT.gov:

ClinicalTrials.gov

FDA:

Food and Drug Administration

FDAAA:

Food and Drug Administration Amendments Act

ICMJE:

International Committee of Medical Journal Editors

ISCRCTN:

International Standard Randomised Controlled Trials Number

IF:

impact factor

PEDro:

Physiotherapy Evidence Database

RCT:

randomized controlled trial

SR:

systematic review

WHO ICTRP:

World Health Organization International Clinical Trials Registry Platform

References

  1. 1.

    Zarin DA, Tse T. Medicine. Moving toward transparency of clinical trials. Science. 2008;319(5868):1340–2.

  2. 2.

    De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJ, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004;351(12):1250–1.

  3. 3.

    Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gotzsche PC, Krumholz HM, Ghersi D, van der Worp HB. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66.

  4. 4.

    Topol EJ. Failing the public health--rofecoxib, Merck, and the FDA. N Engl J Med. 2004;351(17):1707–9.

  5. 5.

    Juni P, Nartey L, Reichenbach S, Sterchi R, Dieppe PA, Egger M. Risk of cardiovascular events and rofecoxib: cumulative meta-analysis. Lancet. 2004;364(9450):2021–9.

  6. 6.

    Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R, Thompson MJ, Spencer EA, Onakpoya I, Mahtani KR, Nunan D, et al. Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database Syst Rev. 2014;(4):CD008965.

  7. 7.

    Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361(20):1963–71.

  8. 8.

    Vedula SS, Goldman PS, Rona IJ, Greene TM, Dickersin K. Implementation of a publication strategy in the context of reporting biases. A case study based on new documents from Neurontin litigation. Trials. 2012;13:136.

  9. 9.

    Krleza-Jeric K, Chan AW, Dickersin K, Sim I, Grimshaw J, Gluud C. Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1). BMJ. 2005;330(7497):956–8.

  10. 10.

    Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med. 2010;153(3):158–66.

  11. 11.

    Chan AW, Pello A, Kitchen J, Axentiev A, Virtanen JI, Liu A, Hemminki E. Association of Trial Registration With Reporting of Primary Outcomes in Protocols and Publications. JAMA. 2017;318(17):1709-1711. https://doi.org/10.1001/jama.2017.13001.

  12. 12.

    Zarin DA, Tse T, Williams RJ, Carr S. Trial reporting in ClinicalTrials.gov - the Final Rule. N Engl J Med. 2016;375(20):1998–2004.

  13. 13.

    Hudson KL, Lauer MS, Collins FS. Toward a new era of trust and transparency in clinical trials. JAMA. 2016;316(13):1353–4.

  14. 14.

    Viergever RF, Li K. Trends in global clinical trial registration: an analysis of numbers of registered clinical trials in different parts of the world from 2004 to 2013. BMJ Open. 2015;5(9):e008932.

  15. 15.

    Trial Registration, Organizations with Policies. http://www.who.int/ictrp/trial_reg/en/index2.html. Accessed 9 Sept 2018.

  16. 16.

    Laine C, Horton R, DeAngelis CD, Drazen JM, Frizelle FA, Godlee F, Haug C, Hebert PC, Kotzin S, Marusic A, et al. Clinical trial registration: looking back and moving ahead. Lancet. 2007;369(9577):1909–11.

  17. 17.

    Weber WE, Merino JG, Loder E. Trial registration 10 years on. BMJ. 2015;351:h3572.

  18. 18.

    Zarin DA, Tse T, Williams RJ, Rajakannan T. Update on trial registration 11 years after the ICMJE policy was established. N Engl J Med. 2017;376(4):383–91.

  19. 19.

    Chen R, Desai NR, Ross JS, Zhang W, Chau KH, Wayda B, Murugiah K, Lu DY, Mittal A, Krumholz HM. Publication and reporting of clinical trial results: cross sectional analysis across academic medical centers. BMJ. 2016;352:i637.

  20. 20.

    Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ. 2012;344:d7292.

  21. 21.

    Hamm MP, Hartling L, Milne A, Tjosvold L, Vandermeer B, Thomson D, Curtis S, Klassen TP. A descriptive analysis of a representative sample of pediatric randomized controlled trials published in 2007. BMC Pediatr. 2010;10:96.

  22. 22.

    Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302(9):977–84.

  23. 23.

    Missing trial data – briefing notes. http://www.alltrials.net/wp-content/uploads/2013/01/Missing-trials-briefing-note.pdf. Accessed 9 Sept 2018.

  24. 24.

    Dal-Re R, Marusic A. Prevention of selective outcome reporting: let us start from the beginning. Eur J Clin Pharmacol. 2016;72(10):1283–8.

  25. 25.

    Jones CW, Keil LG, Holland WC, Caughey MC, Platts-Mills TF. Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med. 2015;13:282.

  26. 26.

    Bashir R, Bourgeois FT, Dunn AG. A systematic review of the processes used to link clinical trial registrations to their published results. Syst Rev. 2017;6(1):123.

  27. 27.

    Glanville JM, Duffy S, McCool R, Varley D. Searching ClinicalTrials.gov and the International Clinical Trials Registry Platform to inform systematic reviews: what are the optimal search approaches? J Med Libr Assoc. 2014;102(3):177–83.

  28. 28.

    Trikalinos TA, Trow P, Schmid CH. Simulation-based comparison of methods for meta-analysis of proportions and rates. Rockville: Agency for Healthcare Research and Quality (US); 2013.

  29. 29.

    Bakbergenuly I, Kulinskaya E, Morgenthaler S. Inference for binomial probability based on dependent Bernoulli random variables with applications to meta-analysis and group level studies. Biom J. 2016;58(4):896–914.

  30. 30.

    Haidich AB, Ioannidis JP. Effect of early patient enrollment on the time to completion and publication of randomized controlled trials. Am J Epidemiol. 2001;154(9):873–80.

  31. 31.

    Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA. 1998;279(4):281–6.

  32. 32.

    Anand V, Scales DC, Parshuram CS, Kavanagh BP. Registration and design alterations of clinical trials in critical care: a cross-sectional observational study. Intensive Care Med. 2014;40(5):700–22.

  33. 33.

    Boespflug A, Gan H, Chen EX, Pond G, You B. Consistency in the analysis and reporting of PEPs in oncology randomized controlled trials from registration to publication: a systematic review. Bull Cancer. 2012;99(10):943–52.

  34. 34.

    Bonnot B, Yavchitz A, Mantz J, Paugam-Burtz C, Boutron I. Selective primary outcome reporting in high-impact journals of anaesthesia and pain. Br J Anaesth. 2016;117(4):542–3.

  35. 35.

    Bradley HA, Rucklidge JJ, Mulder RT. A systematic review of trial registration and selective outcome reporting in psychotherapy randomized controlled trials. Acta Psychiatr Scand. 2017;135(1):65–77.

  36. 36.

    Byrne JL, Yee T, O’Connor K, Dyson MP, Ball GD. Registration status and methodological reporting of randomized controlled trials in obesity research: a review. Obesity (Silver Spring). 2017;25(4):665–70.

  37. 37.

    Cybulski L, Mayo-Wilson E, Grant S. Improving transparency and reproducibility through registration: the status of intervention trials published in clinical psychology journals. J Consult Clin Psychol. 2016;84(9):753–67.

  38. 38.

    Dechartres A, Ravaud P, Atal I, Riveros C, Boutron I. Association between trial registration and treatment effect estimates: a meta-epidemiological study. BMC Med. 2016;14(1):100.

  39. 39.

    Dekkers OM, Soonawala D, Vandenbroucke JP, Egger M. Reporting of noninferiority trials was incomplete in trial registries. J Clin Epidemiol. 2011;64(9):1034–8.

  40. 40.

    Dekkers OM, Cevallos M, Buhrer J, Poncet A, Ackermann Rau S, Perneger TV, Egger M. Comparison of noninferiority margins reported in protocols and publications showed incomplete and inconsistent reporting. J Clin Epidemiol. 2015;68(5):510–7.

  41. 41.

    El-Boghdadly K, Wiles M, Atton S, Bailey C. Analysis of registration rigour of randomised controlled trials published in anaesthesia. Anaesthesia. 2017;72:17.

  42. 42.

    Emdin C, Odutayo A, Hsiao A, Shakir M, Hopewell S, Rahimi K, Altman DG. Association of cardiovascular trial registration with positive study findings: epidemiological Study of Randomized Trials (ESORT). JAMA Intern Med. 2015;175(2):304–7.

  43. 43.

    Farquhar CM, Showell MG, Showell EAE, Beetham P, Baak N, Mourad S, Jordan VMB. Clinical trial registration in fertility trials - a case for improvement? Hum Reprod. 2017;32(9):1827–34.

  44. 44.

    Farquhar CM, Showell MG, Showell EAE, Beetham P, Baak N, Mourad S, Jordan VMB. Clinical trial registration was not an indicator for low risk of bias. J Clin Epidemiol. 2017;84:47–53.

  45. 45.

    Gates A, Hartling L, Vandermeer B, Caldwell P, Contopoulos-Ioannidis DG, Curtis S, Fernandes RM, Klassen TP, Williams K, Dyson MP. The conduct and reporting of child health research: an analysis of randomized controlled trials published in 2012 and evaluation of change over 5 years. J Pediatr. 2018;193:237–244.e37.

  46. 46.

    Gray R, Badnapurkar A, Hassanein E, Thomas D, Barguir L, Baker C, Jones M, Bressington D, Brown E, Topping A. Registration of randomized controlled trials in nursing journals. Res Integr Peer Rev. 2017;2:8. https://doi.org/10.1186/s41073-017-0036-9.

  47. 47.

    Hardt JL, Metzendorf MI, Meerpohl JJ. Surgical trials and trial registers: a cross-sectional study of randomized controlled trials published in journals requiring trial registration in the author instructions. Trials. 2013;14:407.

  48. 48.

    Jones CW, Platts-Mills TF. Quality of registration for clinical trials published in emergency medicine journals. Ann Emerg Med. 2012;60(4):458–464.e1.

  49. 49.

    Jones PM, Chow JTY, Arango MF, Fridfinnson JA, Gai N, Lam K, Turkstra TP. Comparison of registered and reported outcomes in randomized clinical trials published in anesthesiology journals. Anesth Analg. 2017;125(4):1292–300.

  50. 50.

    Killeen S, Sourallous P, Hunter IA, Hartley JE, Grady HL. Registration rates, adequacy of registration, and a comparison of registered and published primary outcomes in randomized controlled trials published in surgery journals. Ann Surg. 2014;259(1):193–6.

  51. 51.

    Kunath F, Grobe HR, Keck B, Rucker G, Wullich B, Antes G, Meerpohl JJ. Do urology journals enforce trial registration? A cross-sectional study of published trials. BMJ Open. 2011;1(2):e000430.

  52. 52.

    Li XQ, Yang GL, Tao KM, Zhang HQ, Zhou QH, Ling CQ. Comparison of registered and published primary outcomes in randomized controlled trials of gastroenterology and hepatology. Scand J Gastroenterol. 2013;48(12):1474–83.

  53. 53.

    Mann E, Nguyen N, Fleischer S, Meyer G. Compliance with trial registration in five core journals of clinical geriatrics: a survey of original publications on randomised controlled trials from 2008 to 2012. Age Ageing. 2014;43(6):872–6.

  54. 54.

    McGee RG, Su M, Kelly PJ, Higgins GY, Craig JC, Webster AC. Trial registration and declaration of registration by authors of randomized controlled trials. Transplantation. 2011;92(10):1094–100.

  55. 55.

    Milette K, Roseman M, Thombs BD. Transparency of outcome reporting and trial registration of randomized controlled trials in top psychosomatic and behavioral health journals: a systematic review. J Psychosom Res. 2011;70(3):205–17.

  56. 56.

    Nankervis H, Baibergenova A, Williams HC, Thomas KS. Prospective registration and outcome-reporting bias in randomized controlled trials of eczema treatments: a systematic review. J Invest Dermatol. 2012;132(12):2727–34.

  57. 57.

    Norris SL, Holmer HK, Fu R, Ogden LA, Viswanathan MS, Abou-Setta AM. Clinical trial registries are of minimal use for identifying selective outcome and analysis reporting. Res Synth Methods. 2014;5(3):273–84.

  58. 58.

    Odutayo A, Emdin CA, Hsiao AJ, Shakir M, Copsey B, Dutton S, Chiocchia V, Schlussel M, Dutton P, Roberts C, et al. Association between trial registration and positive study findings: cross sectional study (Epidemiological Study of Randomized Trials-ESORT). BMJ. 2017;356:j917.

  59. 59.

    Ostervig RM, Sonne A, Rasmussen LS. Registration of randomized clinical trials--a challenge. Acta Anaesthesiol Scand. 2015;59(8):986–9.

  60. 60.

    Pidgeon TE, Limb C, Agha RA, Whitehurst K, Chandrakumar C, Wellstead G, Fowler AJ, Orgill DP. The use of study registration and protocols in plastic surgery research: a systematic review. Int J Surg. 2017;44:215–22.

  61. 61.

    Pinto RZ, Elkins MR, Moseley AM, Sherrington C, Herbert RD, Maher CG, Ferreira PH, Ferreira ML. Many randomized trials of physical therapy interventions are not adequately registered: a survey of 200 published trials. Phys Ther. 2013;93(3):299–309.

  62. 62.

    Reveiz L, Bonfill X, Glujovsky D, Pinzon CE, Asenjo-Lobos C, Cortes M, Canon M, Bardach A, Comande D, Cardona AF. Trial registration in Latin America and the Caribbean’s: study of randomized trials published in 2010. J Clin Epidemiol. 2012;65(5):482–7.

  63. 63.

    Riehm KE, Azar M, Thombs BD. Transparency of outcome reporting and trial registration of randomized controlled trials in top psychosomatic and behavioral health journals: a 5-year follow-up. J Psychosom Res. 2015;79(1):1–12.

  64. 64.

    Rosati P, Porzsolt F, Ricciotti G, Testa G, Inglese R, Giustini F, Fiscarelli E, Zazza M, Carlino C, Balassone V, et al. Major discrepancies between what clinical trial registries record and paediatric randomised controlled trials publish. Trials. 2016;17(1):430.

  65. 65.

    Scott A, Rucklidge JJ, Mulder RT. Is mandatory prospective trial registration working to prevent publication of unregistered trials and selective outcome reporting? An observational study of five psychiatry journals that mandate prospective clinical trial registration. PLoS One. 2015;10(8):e0133718.

  66. 66.

    Shinohara K, Tajika A, Imai H, Takeshima N, Hayasaka Y, Furukawa TA. Protocol registration and selective outcome reporting in recent psychiatry trials: new antidepressants and cognitive behavioural therapies. Acta Psychiatr Scand. 2015;132(6):489–98.

  67. 67.

    Sims MT, Sanchez ZC, Herrington JM, Hensel JB, Henning NM, Scheckel CJ, Vassar M. Shoulder arthroplasty trials are infrequently registered: a systematic review of trials. PLoS One. 2016;11(10):e0164984.

  68. 68.

    Smail-Faugeron V, Fron-Chabouis H, Durieux P. Clinical trial registration in oral health journals. J Dent Res. 2015;94(3 Suppl):8S–13S.

  69. 69.

    Walker KF, Stevenson G, Thornton JG. Discrepancies between registration and publication of randomised controlled trials: an observational study. JRSM Open. 2014;5(5):2042533313517688.

  70. 70.

    van de Wetering FT, Scholten RJ, Haring T, Clarke M, Hooft L. Trial registration numbers are underreported in biomedical publications. PLoS One. 2012;7(11):e49599.

  71. 71.

    Wiebe J, Detten G, Scheckel C, Gearhart D, Wheeler D, Sanders D, Vassar M. The heart of the matter: outcome reporting bias and registration status in cardio-thoracic surgery. Int J Cardiol. 2017;227:299–304.

  72. 72.

    You B, Gan HK, Pond G, Chen EX. Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: a systematic review. J Clin Oncol. 2012;30(2):210–6.

  73. 73.

    Azar M, Riehm KE, McKay D, Thombs BD. Transparency of outcome reporting and trial registration of randomized controlled trials published in the journal of consulting and clinical psychology. PLoS One. 2015;10(11):e0142894.

  74. 74.

    De Oliveira GS Jr, Jung MJ, McCarthy RJ. Discrepancies between randomized controlled trial registry entries and content of corresponding manuscripts reported in anesthesiology journals. Anesth Analg. 2015;121(4):1030–3.

  75. 75.

    Wager E, Williams P, Project Overcome failure to Publish nEgative fiNdings C. “Hardly worth the effort”? Medical journals’ policies and their editors’ and publishers’ views on trial registration and publication bias: quantitative and qualitative study. BMJ. 2013;347:f5248.

  76. 76.

    Hooft L, Korevaar DA, Molenaar N, Bossuyt PM, Scholten RJ. Endorsement of ICMJE’s Clinical Trial Registration Policy: a survey among journal editors. Neth J Med. 2014;72(7):349–55.

  77. 77.

    Moorthy VS, Karam G, Vannice KS, Kieny MP. Rationale for WHO’s new position calling for prompt reporting and public disclosure of interventional clinical trial results. PLoS Med. 2015;12(4):e1001819.

  78. 78.

    Pica N, Bourgeois F. Discontinuation and Nonpublication of Randomized Clinical Trials Conducted in Children. Pediatrics. 2016;138(3). pii: e20160223. https://doi.org/10.1542/peds.2016-0223.

  79. 79.

    Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database--update and key issues. N Engl J Med. 2011;364(9):852–60.

  80. 80.

    Dickersin K, Rennie D. The evolution of trial registries and their use to assess the clinical trial enterprise. JAMA. 2012;307(17):1861–4.

  81. 81.

    Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias G. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS One. 2013;8(7):e66844.

  82. 82.

    Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, Williamson PR, Kirkham JJ. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11(6):e1001666.

  83. 83.

    Loder E, Loder S, Cook S. Characteristics and publication fate of unregistered and retrospectively registered clinical trials submitted to The BMJ over 4 years. BMJ Open. 2018;8(2):e020037.

  84. 84.

    Mayo-Wilson E, Heyward J, Keyes A, Reynolds J, White S, Atri N, Alexander GC, Omar A, Ford DE, National Clinical Trials R, et al. Clinical trial registration and reporting: a survey of academic organizations in the United States. BMC Med. 2018;16(1):60.

  85. 85.

    O’Reilly EK, Hassell NJ, Snyder DC, Natoli S, Liu I, Rimmler J, Amspacher V, Burnett BK, Parrish AB, Berglund JP, et al. ClinicalTrials.gov reporting: strategies for success at an academic health center. Clin Transl Sci. 2015;8(1):48–51.

  86. 86.

    Goldacre B. How to get all trials reported: audit, better data, and individual accountability. PLoS Med. 2015;12(4):e1001821.

  87. 87.

    Meerpohl JJ, Schell LK, Bassler D, Gallus S, Kleijnen J, Kulig M, La Vecchia C, Marusic A, Ravaud P, Reis A, et al. Evidence-informed recommendations to reduce dissemination bias in clinical research: conclusions from the OPEN (Overcome failure to Publish nEgative fiNdings) project based on an international consensus meeting. BMJ Open. 2015;5(5):e006666.

  88. 88.

    Doernberg SN, Wendler D. Ensuring respect for human research participants: institutional review boards and sharing results from research. JAMA. 2016;316(11):1149–50.

  89. 89.

    The COMPare Trials Project. www.COMPare-trials.org. Accessed 9 Sept 2018.

Download references

Funding

FTB and AGD have received grant support from the Agency for Healthcare Research and Quality (R03HS024798 developing methods to improve systematic reviews using clinical trial registries). FTB has received grant support from the National Institutes of Health and holds an Innovation in Regulatory Science Award from the Burroughs Wellcome Fund. AGD has received grant support from the National Health and Medical Research Council (Australia) for unrelated research.

Availability of data and materials

All data generated or analyzed during this study are included in this published article [and its supplementary information files].

Author information

Affiliations

  1. Department of Biostatistics, Boston University School of Public Health, Boston, Massachusetts, USA

    • Ludovic Trinquart

  2. Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia

    • Adam G. Dunn

  3. Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, USA

    • Florence T. Bourgeois

  4. Center for Pediatric Therapeutics and Regulatory Science, and Computational Health Informatics Program, Boston Children’s Hospital, Boston, MA, USA

    • Florence T. Bourgeois

Authors

  1. Search for Ludovic Trinquart in:

  2. Search for Adam G. Dunn in:

  3. Search for Florence T. Bourgeois in:

Contributions

LT and FTB conceived and designed the study. LT, AGD, and FTB selected the studies and collected the data. LT analyzed data, and all authors interpreted the results. LT drafted and revised the paper. AGD and FTB revised the draft paper. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Ludovic Trinquart.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

abstracted data. (CSV 2 kb)

Additional file 2:

R code. (HTML 1110 kb)

Additional file 3:

PRISMA checklist. (DOCX 149 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Keywords

  • Randomized controlled trials
  • Registration
  • Reporting bias

BMC Medicine

ISSN: 1741-7015

Contact us