Skip to main content
  • Research article
  • Open access
  • Published:

Is the endocrine research pipeline broken? A systematic evaluation of the Endocrine Society clinical practice guidelines and trial registration

Abstract

Background

Very low quality (VLQ) evidence translates into very low confidence in the balance of risk and benefits based on the estimates drawn from the body of evidence. Consequently, this assessment highlights gaps in the research evidence, i.e. knowledge gaps, for important clinical questions. In this way, expert guideline panels identify priority knowledge gaps that, arguably, should inform the research agenda and prioritize scarce research economical resources. The extent to which the research agenda reflects the knowledge gaps identified in clinical practice guidelines is unknown.

Methods

A systematic evaluation of the Endocrine Society (ES) clinical practice guidelines portfolio from 2008 to 2014 was conducted with the objectives to identify (1) recommendations in the ES clinical practice guidelines based on VLQ evidence reflecting knowledge gaps in endocrinology, and (2) active research designed to address these gaps by searching the clinical trial registry, clinicaltrials.gov, using terms describing patients (diseases), interventions, comparison, and outcomes.

Results

In 25 ES guidelines, we found 660 recommendations, of which 131 (20 %) were supported by VLQ evidence. Clinical trialists are attempting to answer 28 (21 %) of these knowledge gaps by performing 69 clinical trials.

Conclusion

The research enterprise is addressing one in five knowledge gaps identified in clinical practice recommendations in endocrinology. These findings suggest an inefficiency in the allocation of very scarce research economical resources. Linking the research agenda to evidence gaps in clinical practice guidelines may improve both the efficiency of the research enterprise and the translation of evidence into more confident clinical practice.

Peer Review reports

Background

The Endocrine Society (ES) has created clinical practice guidelines to aid clinicians in the care of patients with endocrine disorders. In 2005, the ES adopted the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Group system [14]. The GRADE approach rates the panel’s confidence in the risk estimates of favorable and unfavorable outcomes. This confidence in the estimates is captured by classifying the quality of the body of evidence supporting a recommendation into one of four categories: high, moderate, low, and very low quality (VLQ) [14]. VLQ evidence results when the body of evidence is comprised of studies at high risk of bias, yielding imprecise results or results of indirect relevance to the recommendation, and/or results that are inconsistent across studies or are not fully reported. VLQ evidence translates into very low confidence in the balance of risk and benefits based on the estimates drawn from the body of evidence [3, 4]. Consequently, this assessment highlights gaps in the research evidence, i.e. knowledge gaps, for important clinical questions. Since these knowledge gaps affect the assessments about the balance of benefits and harms of alternative courses of action they reduce our confidence that patients will be better off were they to receive care consistent with recommendations based on VLQ evidence.

A recent study showed that low to very low quality evidence, largely derived from small observational studies, supported most endocrinology guideline recommendations [5, 6]. To this extent, guidelines that explicitly account for confidence in estimates from the body of evidence, such as ES guidelines, can be used to identify important knowledge gaps and guide the research agenda. This is particularly important in the face of scarce, research dollars. Moreover, it has been previously estimated that 85 % of research is of low impact or wasted mainly due to being unnecessary, poorly designed, biased, unusable, incompletely published, or simply addressing the wrong research question. A better connection between knowledge gaps identified in practice guidelines and the research enterprise could reduce research waste [712]. To explore the integrity of the pipeline connecting knowledge gaps with ongoing research, we conducted a study using ES guidelines.

Methods

We conducted a systematic evaluation of the available ES clinical practice guidelines to identify clinical recommendations that are based on VLQ evidence and that potentially reflect knowledge gaps in endocrinology. Using the ES guideline web site, we identified and retrieved all ES clinical practice guidelines issued from 2008 to 2014 [1338]. For each guideline, two reviewers working independently searched and extracted the number of graded recommendations in each guideline and those rated as based on VLQ evidence. Guideline panels following the GRADE approach, as is the case with ES guidelines, rate evidence as VLQ when the body of evidence produces estimates about which we have very low confidence. Studies produce low-confidence estimates when they are at high risk of bias, produce results that are of indirect relevance to the recommendation, imprecise or inconsistent, or when there is evidence of incomplete and biased reporting [3, 4, 39].

Because of problems with classification, some recommendations based on VLQ evidence should not have been graded as they represent best practice statements in which there is no sensible alternative. In this sense, best practice recommendations are thought to do substantially more good than harm (or vice versa) and therefore no one would consider doing a study to definitively establish the answer to the implicit question [40]. Examples of best practice recommendations include: (1) “We suggest that female-to-male transsexual persons evaluate the risks and benefits of including total hysterectomy and oophorectomy as part of sex reassignment surgery”, and (2) “In patients presenting with heart failure, initial assessment should be made of the patient’s ability to perform routine/desired activities of daily living” [20, 38, 41]. After excluding these, the crude inter-observer agreement for the identification of recommendations based of VLQ evidence was 96 %.

We then identified the research questions relevant to these knowledge gaps in terms of patients, interventions, comparisons, and outcomes (PICO). For each VLQ evidence item, we drafted a research question that a clinical trial could answer using the PICO format (e.g. Patients – patients with large adrenal pheochromocytomas; Intervention – minimally invasive adrenalectomy; Comparison – open resection adrenalectomy; Outcome – complete tumor resection, tumor rupture avoidance, and local recurrence rates) [1, 35, 39]. The objective of this step was to make the underlying questions explicit thus ensuring reproducibility of our methods. To calibrate this process, two researchers independently produced these questions for 11 recommendations with an initial agreement of 80 %. This process was repeated until the agreement was 100 %, achieved after 20 recommendations.

We then searched the clinicaltrials.gov database for active studies (randomized and observational) addressing the questions identified from the guidelines. Since 2000, this registry has emerged as the most complete including trials from 188 countries. A clinical trial was deemed eligible if it addressed the PICO question with at least one of the necessary outcomes. For each eligible clinical trial, we extracted the study design, source of funding, year of entry, and the sample size. Although we did not look for publication of results, we excluded trials completed 5 or more years prior to the date of guideline publication. Reviewers working independently searched the clinical trial registry until 100 % agreement, a point reached after searching for 20 questions. A single reviewer completed the search for the rest of the questions.

Statistical analysis

We performed a descriptive analysis and summarized continuous variables as mean (SD) and presented percentages in cases of categorical variables.

Results

We identified 25 Clinical Practice Guidelines from the ES with 660 recommendations, of which 209 (32 %) were supported by VLQ evidence [832]. After excluding 78 (12 %) best practice statements, the total was 131 (20 %) recommendations based on VLQ evidence (Fig. 1). The majority of the guidelines supported the care of patients with pituitary, gonadal, and adrenal disorders [13], and most recommendations supported by VLQ evidence came from these guidelines (24 %; Fig. 2).

Fig. 1
figure 1

Selection process. ES, Endocrine Society; VLQ, Very low quality

Fig. 2
figure 2

Proportion of recommendations based on VLQ evidence with and without active research studies. VLQ, Very low quality; Rec., Recommendation; AR., Active research; PAG, Pituitary adrenal and gonad axis; CVD, Cardiovascular disease. Numbers in percentage correspond to recommendations based on VLQ evidence with active research

Active research was identified for 28 (21 %) of these 131 recommendations represented by 69 clinical studies (mean of 2.5 studies per each recommendation – when taking into consideration those with active research; Fig. 1). Of these, 35 (51 %) were randomized trials and 34 (49 %) were observational. Thirteen reported industry funding and six were multicenter studies. Of the 69 active studies, 42 (60 %) were in the guidelines of thyroid dysfunction during pregnancy, testosterone therapy in adult men, and diagnosis of Cushing syndrome which had a total of 32 recommendations supported by VLQ evidence, of which 15 had at least one active research study.

Most of the identified clinical trials addressed knowledge gaps affecting recommendations for patients with thyroid disorders (70 %); the least dealt with were gaps in the evidence for care of patients with diabetes, obesity, and cardiovascular disease (16 %; Fig. 2).

Discussion

Important knowledge gaps are evident in 10 of every 50 ES clinical practice guideline recommendations, of which the research enterprise is actively addressing, at best, two. Moreover, of the active trials, 60 % are trying to improve the quality of evidence in only three of the 25 guidelines. In some cases, several studies are actively addressing the same question. The research enterprise, thus characterized, poorly reflects the knowledge gaps at the frontline of endocrine practice, covering these gaps incompletely and sometimes redundantly. Multiple explanations exist for these observations. The research enterprise may not be aware of these gaps because funding agencies and researchers do not use clinical practice guidelines to identify knowledge gaps, or they may be responding to these gaps with basic studies that are not in the registry. Alternatively, ES expert ratings of confidence in estimates evidence have not been explicitly communicated to researchers and funders to facilitate the development of a practice-relevant research agenda.

This cross-sectional study contributes to the understanding of a seemingly broken research pipeline. However, there are dynamic aspects that only a longitudinal study could answer. For example, in its 2014 version, the ES guideline on Androgen Therapy in Women had seven recommendations based on VLQ evidence, all new since its 2006 version [42]. Conversely, the evidence supporting two of their 2006 recommendations was upgraded in 2014 from warranting very low to warranting low confidence in the estimates [38, 42]. How these changes result from the manner in which the research enterprise responds to knowledge gaps merits further study.

To our knowledge, there is no comprehensive repository of knowledge gaps in endocrinology. This study partially reflects the coverage of the current ES guideline portfolio based on our selection of recommendations supported by VLQ evidence, instead of the much larger set of recommendations based on low quality evidence. A key advantage of the portfolio that enabled this study is the ES’s early adoption of GRADE, which enables the separation of the rating of evidence (used here) from the grading of the strength of recommendation [4]. Further, we used the clinicaltrials.gov registry which, despite being the largest, may have missed research (e.g. well-designed observational studies, not registered studies), particularly outside the United States. Additionally, studies published after the publication of the clinical guidelines that could have assessed these gaps were not part of our analysis. Nevertheless, we identified knowledge gaps in the ES clinical practice guidelines in duplicate and performed a search of clinical trials to address them after high reproducibility was achieved between each of the independent reviewers.

We are also aware that, even though panels may be the best way to identify research gaps, it could be easily argued that this does not automatically mean that trials should be conducted in those areas, e.g. the needed studies might be prohibitively expensive, research gaps might be difficult to study, or the science field is not developed enough to support the conduct of clinical trials. On the other hand, panels choose areas of clinical relevance to formulate recommendations. Often, these represent ongoing practice, which is unlikely to be supported solely by mechanistic hypotheses or basic science data. While we do not know the extent to which our findings apply to other areas of medicine, they may very well represent a trigger to examine the research pipeline and improve the quality and relevance of the evidence. A functional pipeline connecting evidence to recommendations and back can ultimately better support decision making by patients, clinicians, policy makers, and funding agencies.

Conclusion

Researchers are addressing only one in five knowledge gaps identified in clinical practice recommendations in endocrinology. Linking the research agenda to evidence gaps in guidelines may improve both the efficiency of the research enterprise and the translation of evidence into practice by increasing the value and reducing the waste in research.

Abbreviations

ES:

Endocrine Society

GRADE:

Grading of Recommendations, Assessment, Development and Evaluation System

PICO:

Patient, Intervention/Exposure, Comparison and Outcome

VLQ:

Very low quality

References

  1. Guyatt GH, Oxman AD, Schünemann HJ, Tugwell P, Knottnerus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol. 2011;64:380–2.

    Article  PubMed  Google Scholar 

  2. GRADE online learning modules. McMaster University. http://cebgrade.mcmaster.ca/. Accessed April 16, 2015.

  3. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: Rating the quality of evidence. J Clin Epidemiol. 2011;64:401–6.

    Article  PubMed  Google Scholar 

  4. Swiglo BA, Murad MH, Schünemann HJ, Kunz R, Vigersky RA, Guyatt GH, et al. A case for clarity, consistency, and helpfulness: state-of-the-art clinical practice guidelines in endocrinology using the grading of recommendations, assessment, development, and evaluation system. J Clin Endocrinol Metab. 2008;93:666–73.

    Article  CAS  PubMed  Google Scholar 

  5. Brito JP, Domecq JP, Murad MH, Guyatt GH, Montori VM. The Endocrine Society guidelines: when the confidence cart goes before the evidence horse. J Clin Endocrinol Metab. 2013;98:3246–52. doi:10.1210/jc.2013-1814.

    Article  CAS  PubMed  Google Scholar 

  6. Hazlehurst JM, Armstrong MJ, Sherlock M, Rowe IA, O’Reilly MW, Franklyn JA, et al. A comparative quality assessment of evidence-based clinical guidelines in endocrinology. Clin Endocrinol (Oxf). 2013;78:183–90. doi:10.1111/j.1365-2265.2012.04441.x.

    Article  Google Scholar 

  7. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374:86–9.

    Article  PubMed  Google Scholar 

  8. Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383:156–65.

    Article  PubMed  Google Scholar 

  9. Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383:166–75.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Al-Shahi Salman R, Beller E, Kagan J, Hemminki E, Phillips RS, Savulescu J, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014;383:176–85.

    Article  PubMed  Google Scholar 

  11. Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383:257–66.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383:267–76.

    Article  PubMed  Google Scholar 

  13. The Endocrine Society. The Endocrine Society published guidelines. 2014. https://www.endocrine.org/education-and-practice-management/clinical-practice-guidelines.

  14. Martin KA, Chang RJ, Ehrmann DA, Ibanez L, Lobo RA, Rosenfield RL, et al. Evaluation and treatment of hirsutism in premenopausal women: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2008;93:1105–20.

    Article  CAS  PubMed  Google Scholar 

  15. Nieman LK, Biller BM, Findling JW, Newell-Price J, Savage MO, Stewart PM, et al. The diagnosis of Cushing’s syndrome: an Endocrine Society Clinical Practice Guideline. J Clin Endocrinol Metab. 2008;93:1526–40.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Funder JW, Carey RM, Fardella C, Gomez-Sanchez CE, Mantero F, Stowasser M, et al. Endocrine Society. Case detection, diagnosis, and treatment of patients with primary aldosteronism: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2008;93:3266–81.

    Article  CAS  PubMed  Google Scholar 

  17. Rosenzweig JL, Ferrannini E, Grundy SM, Haffner SM, Heine RJ, Horton ES, et al. Primary prevention of cardiovascular disease and type 2 diabetes in patients at metabolic risk: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2008;93:3671–89.

    Article  CAS  PubMed  Google Scholar 

  18. August GP, Caprio S, Fennoy I, Freemark M, Kaufman FR, Lustig RH, et al. Endocrine Society. Prevention and treatment of pediatric obesity: an endocrine society clinical practice guideline based on expert opinion. J Clin Endocrinol Metab. 2008;93:4576–99.

    Article  CAS  PubMed  Google Scholar 

  19. Cryer PE, Axelrod L, Grossman AB, Heller SR, Montori VM, Seaquist ER, et al. Endocrine Society. Evaluation and management of adult hypoglycemic disorders: an Endocrine Society Clinical Practice Guideline. J Clin Endocrinol Metab. 2009;94:709–28.

    Article  CAS  PubMed  Google Scholar 

  20. Hembree WC, Cohen-Kettenis P, de Delemarre-van de Waal HA, Gooren LJ, Meyer 3rd WJ, Spack NP, et al. Endocrine Society. Endocrine treatment of transsexual persons: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2009;94:3132–54.

    Article  CAS  PubMed  Google Scholar 

  21. Bhasin S, Cunningham GR, Hayes FJ, Matsumoto AM, Snyder PJ, Swerdloff RS, et al. Task Force, Endocrine Society. Testosterone therapy in men with androgen deficiency syndromes: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2010;95:2536–59. doi:10.1210/jc.2009-2354.

    Article  CAS  PubMed  Google Scholar 

  22. Speiser PW, Azziz R, Baskin LS, Ghizzoni L, Hensle TW, Merke DP, et al. Endocrine Society. Congenital adrenal hyperplasia due to steroid 21-hydroxylase deficiency: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2010;95:4133–60.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Melmed S, Casanueva FF, Hoffman AR, Kleinberg DL, Montori VM, Schlechte JA, et al. Endocrine Society. Diagnosis and treatment of hyperprolactinemia: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2011;96:273–88. doi:10.1210/jc.2010-1692.

    Article  CAS  PubMed  Google Scholar 

  24. Heber D, Greenway FL, Kaplan LM, Livingston E, Salvador J, Still C. Endocrine Society. Endocrine and nutritional management of the post-bariatric surgery patient: an Endocrine Society Clinical Practice Guideline. J Clin Endocrinol Metab. 2010;95:4823–43.

    Article  CAS  PubMed  Google Scholar 

  25. Freda PU, Beckers AM, Katznelson L, Molitch ME, Montori VM, Post KD, et al. Endocrine Society. Pituitary incidentaloma: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2011;96:894–904. doi:10.1210/jc.2010-1048.

    Article  CAS  PubMed  Google Scholar 

  26. Molitch ME, Clemmons DR, Malozowski S, Merriam GR, Vance ML, Endocrine Society. Evaluation and treatment of adult growth hormone deficiency: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2011;96:1587–609. doi:10.1210/jc.2011-0179.

    Article  CAS  PubMed  Google Scholar 

  27. Klonoff DC, Buckingham B, Christiansen JS, Montori VM, Tamborlane WV, Vigersky RA, et al. Endocrine Society. Continuous glucose monitoring: an Endocrine Society Clinical Practice Guideline. J Clin Endocrinol Metab. 2011;96:2968–79.

    Article  CAS  PubMed  Google Scholar 

  28. Holick MF, Binkley NC, Bischoff-Ferrari HA, Gordon CM, Hanley DA, Heaney RP, et al. Endocrine Society. Evaluation, treatment, and prevention of vitamin D deficiency: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2011;96:1911–30.

    Article  CAS  PubMed  Google Scholar 

  29. Umpierrez GE, Hellman R, Korytkowski MT, Kosiborod M, Maynard GA, Montori VM, et al. Endocrine Society. Management of hyperglycemia in hospitalized patients in non-critical care setting: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2012;97:16–38.

    Article  CAS  PubMed  Google Scholar 

  30. Watts NB, Adler RA, Bilezikian JP, Drake MT, Eastell R, Orwoll ES, et al. Endocrine Society. Osteoporosis in men: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2012;97:1802–22.

    Article  CAS  PubMed  Google Scholar 

  31. De Groot L, Abalovich M, Alexander EK, Amino N, Barbour L, Cobin RH, et al. Management of thyroid dysfunction during pregnancy and postpartum: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2012;97:2543–65.

    Article  PubMed  Google Scholar 

  32. Berglund L, Brunzell JD, Goldberg AC, Goldberg IJ, Sacks F, Murad MH, et al. Endocrine society. Evaluation and treatment of hypertriglyceridemia: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2012;97:2969–89.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Blumer I, Hadar E, Hadden DR, Jovanovič L, Mestman JH, Murad MH, et al. Diabetes and pregnancy: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2013;98:4227–49. doi:10.1210/jc.2013-2465.

    Article  CAS  PubMed  Google Scholar 

  34. Legro RS, Arslanian SA, Ehrmann DA, Hoeger KM, Murad MH, Pasquali R, et al. Endocrine Society. Diagnosis and treatment of polycystic ovary syndrome: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2013;98:4565–92.

    Article  CAS  PubMed  Google Scholar 

  35. Lenders JW, Duh QY, Eisenhofer G, Gimenez-Roqueplo AP, Grebe SK, Murad MH, et al. Endocrine Society. Pheochromocytoma and paraganglioma: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2014;99:1915–42.

    Article  CAS  PubMed  Google Scholar 

  36. Katznelson L, Laws Jr ER, Melmed S, Molitch ME, Murad MH, Utz A, et al. Endocrine Society. Acromegaly: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2014;99:3933–51.

    Article  CAS  PubMed  Google Scholar 

  37. Singer FR, Bone 3rd HG, Hosking DJ, Lyles KW, Murad MH, Reid IR, et al. Paget’s disease of bone: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2014;99:4408–22.

    Article  CAS  PubMed  Google Scholar 

  38. Wierman ME, Arlt W, Basson R, Davis SR, Miller KK, Murad MH, et al. Androgen therapy in women: a reappraisal: an Endocrine Society clinical practice guideline. Clin Endocrinol Metab. 2014;99:3489–510.

    Article  CAS  Google Scholar 

  39. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64:383–94.

    Article  PubMed  Google Scholar 

  40. Guyatt GH, Schünemann HJ, Djulbegovic B, Akl EA. Guideline panels should not GRADE good practice statements. J Clin Epidemiol. 2015;68:597–600.

    Article  PubMed  Google Scholar 

  41. Hunt SA, Abraham WT, Chin MH, Feldman AM, Francis GS, Ganiats TG, et al. 2009 focused update incorporated into the ACC/AHA 2005 Guidelines for the Diagnosis and Management of Heart Failure in Adults: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines: developed in collaboration with the International Society for Heart and Lung Transplantation. Circulation. 2009;119:e391–479.

    Article  PubMed  Google Scholar 

  42. Wierman ME, Basson R, Davis SR, Khosla S, Miller KK, Rosner W, et al. Androgen therapy in women: an Endocrine Society Clinical Practice guideline. J Clin Endocrinol Metab. 2006;91:3697–710.

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Victor M. Montori.

Additional information

Competing interests

The authors declare that they have no competing (financial and non-financial) interests.

Authors’ contributions

NSO, RRG, and VMM designed the study, wrote the protocol, served as overall principal investigators, and wrote and reviewed the manuscript. JPB and WFY helped design the study, wrote and made a critical review of the manuscript, and assisted with protocol adaptations and submissions. NSO, RRG, and VMM are guarantors. All authors read and approved the final manuscript.

Naykky Singh Ospina and Rene Rodriguez-Gutierrez contributed equally to this work.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh Ospina, N., Rodriguez-Gutierrez, R., Brito, J.P. et al. Is the endocrine research pipeline broken? A systematic evaluation of the Endocrine Society clinical practice guidelines and trial registration. BMC Med 13, 187 (2015). https://doi.org/10.1186/s12916-015-0435-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12916-015-0435-z

Keywords