Commentary
Open Access

Better prioritization to increase research value and decrease waste

BMC Medicine201513:244

https://doi.org/10.1186/s12916-015-0492-3

©  Dechartres and Ravaud. 2015

Received: 10 September 2015

Accepted: 17 September 2015

Published: 29 September 2015

Abstract

In a recent study published in BMC Medicine, Singh Ospina and colleagues outlined the important gaps between ongoing research and research needs in the field of endocrinology. Many recommendations from clinical practice guidelines are based on a low level of evidence, thereby resulting in research gaps. Despite the publication of around 25,000 randomized controlled trials each year, ongoing research does not cover most of these gaps. In contrast, trials are planned when sufficient data are already available for decision making, which results in redundant research and exposes patients to unnecessary risks. This lack of prioritization contributes to the enormous problem of waste in research. A systematic approach to accumulate the available body of evidence is necessary to determine when we have sufficient evidence and when we have knowledge gaps, defined as research questions with no or a low level of evidence available. Systematic registration of research gaps and their prioritization may help to organize future research. Some initiatives exist, but they need to be generalized.

Please see related research: http://www.biomedcentral.com/1741-7015/13/187

Keywords

Randomized controlled trials Knowledge gaps Planning Research agenda Waste

Background

Waste related to poor planning and prioritization of research

In 2009, Chalmers and Glasziou highlighted the enormous problem of waste in research, estimating that up to 85 % of research investment is wasted [1]. Waste occurs at all stages of research [2–7] and particularly affects planning and prioritization. We have increasing evidence that many trials address low-priority questions that are poorly related to the burden of disease [8] and patient or physician needs [9, 10], do not address patient-important outcomes [11] or use an inadequate comparator. For example, in rheumatology, few trials compare biologically active drugs against each other; comparisons against placebo represent 80 % of trials registered at ClinicalTrials.gov [12]. This lack of head-to-head trials does not allow for answering the pragmatic question raised by patients and their physicians: for this particular disease, which treatment is most effective? Also, it exposes patients to unnecessary risks [12, 13]. Last but not least, trials are frequently planned regardless of the existing evidence. More than 50 % of trial protocols do not refer to systematic reviews [14]. Many trials are planned when sufficient data are already available for decision making, which results in redundant research and exposes patients to unnecessary risks [3]. In contrast, trials are not planned when they are needed to fill research gaps, as highlighted by Singh Ospina and colleagues in a recent study published in BMC Medicine [15]. The authors defined research gaps as clinical questions with a very low level of evidence according to the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach from clinical practice guidelines [15].

The worrying proportion of clinical practice guidelines based on a poor level of evidence

In many clinical practice guidelines, few of the recommendations are based on high-level evidence [16–18]. For example, a 2009 study published in JAMA showed that 11 % of the recommendations in clinical practice guidelines from the American College of Cardiology (ACC) and the American Heart Association (AHA) were considered to be based on a high level of evidence [18]. Identifying clinical questions for which only low-level evidence is available should, in theory, help in planning future clinical trials focusing on these areas. Nevertheless, a study evaluating practice guidelines from the Infectious Diseases Society of America showed no improvement in proportion of recommendations with a high level of evidence over time [17]. It is very worrying that, despite the publication of around 25,000 randomized controlled trials each year, clinical practice guidelines continue to rely mostly on a poor level of evidence, with potentially serious consequences for patient care [19]. Singh Ospina and colleagues used ClinicalTrials.gov to assess the response in terms of new, active studies conducted for research questions with a very low quality of evidence according to the Endocrine Society clinical practice guidelines [15]. The authors found active studies for only one of five recommendations, which suggests that ongoing research does not sufficiently adapt to fill knowledge gaps in endocrinology.

Identifying gaps to decrease waste

The research community is becoming increasingly concerned by these issues. We need to add incremental value to existing evidence by a better connection to future research. A first step is to systematically identify research gaps. As outlined by Singh Ospina and colleagues, clinical practice guidelines could be helpful. In the same way, systematic reviews, by synthesizing the available body of evidence, have a key role to play. The Cochrane Collaboration clearly recommends that review authors systematically comment on the need for further research in a separate section of the review, called “Implications for research” [20]. Then, a second step would be to record research gaps. Some initiatives already exist. The UK Database of Uncertainties about the Effects of Treatments (UK DUETs), established by the National Institute for Health and Care Excellence (NICE), publishes treatment uncertainties reported by patients and clinicians and derived from research recommendations and systematic reviews [21]. The Agency for Health Research and Quality (AHRQ) has developed an approach to identify and prioritize future research needs to be used by researchers and funders to help improve the body of comparative effectiveness evidence that would be useful for decision makers [22]. However, there is not enough information about these burgeoning initiatives. Nothing is done to facilitate the registration of research gaps from different resources and there is no particular incentive for review authors and those of clinical practice guidelines to do so. A joint initiative to centralize registration of gaps in a simple and comprehensible way would be very helpful to enhance communication between researchers, physicians and funders. Finally, the response in terms of new active studies conducted should be monitored to assess the adequacy between ongoing research and knowledge gaps. With the requirement to register trials at ClinicalTrials.gov or in other registries, assessing the clinical trial enterprise and monitoring whether ongoing research fits with research needs has become easier.

Conclusions

Better prioritization of future research is necessary to increase research value in a context of limited human and monetary resources. Some initiatives exist to register and prioritize research gaps. Such efforts should be encouraged and generalized to realign future studies with the existing body of evidence.

Abbreviations

ACC: 

American College of Cardiology

AHA: 

American Heart Association

AHRQ: 

Agency for Health Research and Quality

NICE: 

National Institute for Health and Care Excellence

UK DUETs: 

UK Database of Uncertainties about the Effects of Treatments

Declarations

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Centre de Recherche Epidémiologie et Statistique, INSERM U1153
(2)
Centre d’Epidémiologie Clinique, Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris
(3)
Faculté de Médecine, Université Paris Descartes, Sorbonne Paris Cité
(4)
French Cochrane Centre
(5)
Department of Epidemiology, Mailman School of Public Health, Columbia University

References

  1. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374:86–9.View ArticlePubMedGoogle Scholar
  2. Al-Shahi Salman R, Beller E, Kagan J, Hemminki E, Phillips RS, Savulescu J, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014;383:176–85.View ArticlePubMedGoogle Scholar
  3. Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gulmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383:156–65.View ArticlePubMedGoogle Scholar
  4. Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gotzsche PC, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383:257–66.View ArticlePubMedPubMed CentralGoogle Scholar
  5. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383:267–76.View ArticlePubMedGoogle Scholar
  6. Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383:166–75.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, et al. Biomedical research: increasing value, reducing waste. Lancet. 2014;383:101–4.View ArticlePubMedGoogle Scholar
  8. Emdin CA, Odutayo A, Hsiao AJ, Shakir M, Hopewell S, Rahimi K, et al. Association between randomised trial evidence and global burden of disease: cross sectional study (Epidemiological Study of Randomized Trials--ESORT). BMJ. 2015;350:h117.View ArticlePubMedPubMed CentralGoogle Scholar
  9. Corner J, Wright D, Hopkinson J, Gunaratnam Y, McDonald JW, Foster C. The research priorities of patients attending UK cancer treatment centres: findings from a modified nominal group study. Br J Cancer. 2007;96:875–81.View ArticlePubMedPubMed CentralGoogle Scholar
  10. Tallon D, Chard J, Dieppe P. Relation between agendas of the research community and the research consumer. Lancet. 2000;355:2037–40.View ArticlePubMedGoogle Scholar
  11. Gandhi GY, Murad MH, Fujiyoshi A, Mullan RJ, Flynn DN, Elamin MB, et al. Patient-important outcomes in registered diabetes trials. JAMA. 2008;299:2543–9.View ArticlePubMedGoogle Scholar
  12. Estellat C, Ravaud P. Lack of head-to-head trials and fair control arms: randomized controlled trials of biologic treatment for rheumatoid arthritis. Arch Intern Med. 2012;172:237–44.View ArticlePubMedGoogle Scholar
  13. Ioannidis JP, Karassa FB, Druyts E, Thorlund K, Mills EJ. Biologic agents in rheumatology: unmet issues after 200 trials and $200 billion sales. Nat Rev Rheumatol. 2013;9:665–73.View ArticlePubMedGoogle Scholar
  14. Jones AP, Conroy E, Williamson PR, Clarke M, Gamble C. The use of systematic reviews in the planning, design and conduct of randomised trials: a retrospective cohort of NIHR HTA funded trials. BMC Med Res Methodol. 2013;13:50.View ArticlePubMedPubMed CentralGoogle Scholar
  15. Singh Ospina N, Rodriguez-Gutierrez R, Brito JP, Young Jr WF, Montori VM. Is the endocrine research pipeline broken? A systematic evaluation of the Endocrine Society clinical practice guidelines and trial registration. BMC Med. 2015;13:187.View ArticlePubMedPubMed CentralGoogle Scholar
  16. Hazlehurst JM, Armstrong MJ, Sherlock M, Rowe IA, O’Reilly MW, Franklyn JA, et al. A comparative quality assessment of evidence-based clinical guidelines in endocrinology. Clin Endocrinol (Oxf). 2013;78:183–90.View ArticleGoogle Scholar
  17. Lee DH, Vielemeyer O. Analysis of overall level of evidence behind Infectious Diseases Society of America practice guidelines. Arch Intern Med. 2011;171:18–22.PubMedGoogle Scholar
  18. Tricoci P, Allen JM, Kramer JM, Califf RM, Smith Jr SC. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA. 2009;301:831–41.View ArticlePubMedGoogle Scholar
  19. Heath I. Who’s complacent now? The King’s Fund on general practice. BMJ. 2011;342:d2254.View ArticlePubMedGoogle Scholar
  20. Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions. Version 5.1.0 (updated March 2011). Oxford: The Cochrane Collaboration; 2011. Accessed 23 Sept 2015. http://www.cochrane-handbook.org.Google Scholar
  21. UK Database of Uncertainties about the Effects of Treatments (UK DUETs). Accessed 23 Sept 2015. http://www.library.nhs.uk/duets/
  22. Chang SM, Carey TS, Kato EU, Guise JM, Sanders GD. Identifying research needs for improving health care. Ann Intern Med. 2012;157:439–45.View ArticlePubMedGoogle Scholar

Copyright

© Dechartres and Ravaud. 2015

Advertisement