Skip to main content

Advertisement

  • Editorial
  • Open Access

Reporting guidelines: doing better for readers

BMC Medicine201816:233

https://doi.org/10.1186/s12916-018-1226-0

  • Received: 27 November 2018
  • Accepted: 27 November 2018
  • Published:

Abstract

There is clear guidance on the responsibilities of editors to ensure that the research they publish is of the highest possible quality. Poor reporting is unethical and directly impacts patient care. Reporting guidelines are a relatively recent development to help improve the accuracy, clarity, and transparency of biomedical publications. They have caught on, with hundreds of reporting guidelines now available. Some journals endorse reporting guidelines while a smaller number have used various approaches to implement them. Yet challenges remain – biomedical research is still not optimally reported despite the abundance of reporting guidelines. Electronic algorithms are now being developed to facilitate the choice of correct reporting guideline(s), while other tools are being integrated into journal editorial management processes. Universities need to consider whether it is responsible to advance careers of faculty based on poorly reported research which is of little societal value. If journals embraced auditing of the quality of articles they publish this would give them and their readers essential feedback from which to improve their product.

Keywords

  • Quality of reporting
  • EQUATOR
  • reporting guidelines
  • evidence
  • reproducibility
  • audit and feedback

Background

In 2003, the year when BMC Medicine was first launched, few reporting guidelines existed [1] despite ample evidence of poor reporting across biomedical research. In 1980, Altman [2] noted that the misuse of statistics, and their reporting, was unethical. Later, Rennie, then deputy editor at JAMA, observed “there seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print” [3]. Chalmers and Glasziou [4] reminded us of the substantial and avoidable fiscal waste associated with a plethora of problems of biomedical research, including inadequate reporting. Indeed, bad reporting directly impacts patient care [5, 6].

As BMC Medicine celebrates its 15th anniversary, it is useful to examine whether there are now mechanisms in place to improve the clarity, accuracy, and transparency of publications – a fundamental goal and responsibility of any credible journal [7].

Evolution of better reporting

Using the CONSORT statement as a launching pad, the EQUATOR Network was established in 2006. The vision was to develop a broad basket of tools to help authors, editors, peer reviewers, and others improve the reporting of articles published in biomedicine. The network was formally launched in 2008 [8]. Today, the network is well on the way to meeting its initial remit, with the EQUATOR library being an open repository of more than 400 reporting guidelines developed or currently under development [9]. The network also developed guidance to help others interested in developing a reporting guideline [10], as well as several toolkits for multiple stakeholders, including guidance for authors writing manuscripts, manuscript peer reviewers, and editors wanting to implement reporting guidelines at their journal. All four EQUATOR centers (Australasia, Canada, France, and UK) have publication schools to help authors, particularly early career ones, produce better reports for publication consideration. The algorithm-based EQUATOR Wizard is an initial attempt to help prospective authors identify the most appropriate reporting guideline to use when reporting their research [11]. Other groups, such as the REWARD Alliance (http://rewardalliance.net/), are also drawing attention to these problems and offering solutions. Machine reading tools that provide automatic and immediate assessment of reporting guideline compliance, e.g., CONSORT in the first instance [12], are also starting to appear to help authors and editors. These schemes are now being integrated into editorial management systems and such developments could enable editors invoke quality compliance thresholds below which a manuscript cannot be formally submitted to a journal.

While authors sometimes submit shoddy reports for publication consideration, peer reviewers offer a potential screen of theoretically acceptable publications. Indeed, BMC Medicine has drawn attention to peer review and called for greater professionalism of it [13]. Journals could invoke their own quality threshold by insisting upon using only certified peer reviewers. Peer review supplemented with the use of reporting guidelines is likely to improve the peer-review report and quality of the manuscript under assessment, although more data is required to substantiate this claim. However, it is hard to imagine that using reporting guidelines would provide less informative peer-review reports.

It is possible that the misguided ‘publish or perish’ mantra at academic centers is promoting unscientific and unethical behavior when authors report their research. The prevalence rates of reporting biases are disturbingly high [14, 15], and why researchers would get promoted for such offences is difficult to understand. It is possible that more widespread uptake of declarations of transparency by journals would reduce these reporting biases in publications [16]. Similarly, Universities should consider modifying their incentive criteria towards rewarding career advancement based on better quality publications rather than on quantity. Such a policy directive might also contribute to improving the value of biomedical publications to society.

Challenges

There are now plenty of reporting guidelines to help authors, editors, and peer reviewers, yet several challenges remain. Whether there is reporting guideline inflation resulting in potential confusion for users requires serious reflection. While there is accumulating evidence that use of reporting guidelines is associated with improved reporting, albeit not in all cases [17], this evidence base is limited only to a few reporting guidelines [18, 19]. Additionally, reporting guideline developers seem hesitant to provide this essential data, likely due to the considerable problems in funding such endeavors. However, such as with pharmaceuticals, we should be more cautious about recommending the use of reporting guidelines without evidence of effectiveness. Further, even armed with an initial evidence base about the effectiveness of reporting guidelines, few editors recommend their peer reviewers use them [20]; thus, we need to enhance all implementation efforts [21]. Finally, any attempt to investigate reproducibility is more likely to be possible when reports are clearly and accurately reported; otherwise, such attempts are difficult to initiate [22].

Readers have little information about the quality of what journals publish. Journals need to be more proactive in providing this information such as through regular audits and feedback. Auditing the quality of reports that a journal publishes might provide important information for authors, editors, peer reviewers, and readers in identifying problems and opportunities to enhance the quality of published articles. Making such information available to the public would send a strong positive signal about openness, sharing data, and the journal’s commitment to continuous quality improvement.

Funding programs to improve the quality of reporting biomedical research is remarkably difficult. The irony is that there is volumetric evidence of enormous avoidable waste in the current reporting of biomedical research and why there is hesitancy to fund programs to help improve the quality of reporting is difficult to comprehend. While a few enlightened funders are recognizing the importance of funding such research as well as other journalology and meta-research, there is much to be done to galvanize the majority of funders [23].

Conclusions

Clear, accurate, and transparent reporting of biomedical research remains a considerable problem. Authors, editors, and peer reviewers have failed readers in providing a product that is robust, usable, and reproducible. There are now innovative tools available to help improve this situation, yet we need more active implementation of them by authors, editors, and peer reviewers. University promotion and tenure committees should consider whether offering career advancement based on poorly reported publications is of value to society and ethically sustainable.

Declarations

Acknowledgements

Mr. Raymond Daniel for helping with the management of the references.

Funding

Dr. Moher is supported by a University Research Chair.

Availability of data and materials

Not applicable.

Authors’ contributions

DM conceived and wrote the manuscript. The author read and approved the final manuscript.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

Dr. Moher is the director of the Canadian EQUATOR centre. He is also an author of the cited Declaration of Transparency. Dr. Moher is on the editorial board of BMC Medicine.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Centre for Journalology, Clinical Epidemiology Program, The Ottawa Hospital Research Institute, The Ottawa Hospital - General Campus, 501 Smyth Rd, Room L1288, Ottawa, ON, K1H 8L6, Canada
(2)
School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

References

  1. Moher D, Weeks L, Ocampo M, Seely D, Sampson M, Altman DG, et al. Describing reporting guidelines for health research: a systematic review. J Clin Epidemiol. 2011;64(7):718–42. https://doi.org/10.1016/j.jclinepi.2010.09.013.View ArticlePubMedGoogle Scholar
  2. Altman DG. Statistics and ethics in medical research. Misuse of statistics is unethical. Br Med J. 1980;281(6249):1182–4.View ArticlePubMedPubMed CentralGoogle Scholar
  3. Rennie D. Guarding the guardians: a conference on editorial peer review. JAMA. 1986;256(17):2391–2.Google Scholar
  4. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374:86–9.View ArticlePubMedGoogle Scholar
  5. Duff JM, Leather H, Walden EO, LaPlant KD, George TJ Jr. Adequacy of published oncology randomized controlled trials to provide therapeutic details needed for clinical application. J Natl Cancer Inst. 2010;102:702–5.View ArticlePubMedGoogle Scholar
  6. Dancey JE. From quality of publication to quality of care: translating trials to practice. J Natl Cancer Inst. 2010;102:670–1.View ArticlePubMedGoogle Scholar
  7. World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191–4. https://doi.org/10.1001/jama.2013.281053.View ArticleGoogle Scholar
  8. Thornton H. Report on EQUATOR Network launch meeting 26th June 2008 ‘Achieving Transparency in Reporting Health Research. Int J Surg. 2008;6(6):428–31. https://doi.org/10.1016/j.ijsu.2008.08.002. View ArticlePubMedGoogle Scholar
  9. Enhancing the QUAlity and Transparency Of health Research. http://www.equator-network.org/. Accessed 1 Nov 2018.
  10. Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2):e1000217. https://doi.org/10.1371/journal.pmed.1000217.View ArticlePubMedPubMed CentralGoogle Scholar
  11. The EQUATOR Wizard: A New Tool to Help Authors Find the Right Reporting Guideline. http://www.equator-network.org/2015/12/17/findtherightreportingguideline/. Accessed 1 Nov 2018.
  12. Statreviewer. http://www.statreviewer.com/. Accessed 1 Nov 2018.
  13. Patel J. Why training and specialization is needed for peer review: a case study of peer review for randomized controlled trials. BMC Med. 2014;12:128. https://doi.org/10.1186/s12916-014-0128z.
  14. Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, et al. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11(6):e1001666. https://doi.org/10.1371/journal.pmed.1001666.View ArticlePubMedPubMed CentralGoogle Scholar
  15. Chiu K, Grundy Q, Bero L. Spin’ in published biomedical literature: a methodological systematic review. PLoS Biol. 2017;15(9):e2002173.View ArticlePubMedPubMed CentralGoogle Scholar
  16. Altman DG, Moher D. Declaration of transparency for each research article. BMJ. 2013;347:f4796. https://doi.org/10.1136/bmj.f4796.View ArticlePubMedPubMed CentralGoogle Scholar
  17. Botos J. Reported use of reporting guidelines among JNCI: Journal of the National Cancer Institute authors, editorial outcomes, and reviewer ratings related to adherence to guidelines and clarity of presentation. Res Integr Peer Rev. 2018;3:7. https://doi.org/10.1186/s41073-018-0052-4.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;11:MR000030. https://doi.org/10.1002/14651858.MR000030.pub2. View ArticlePubMedGoogle Scholar
  19. Stevens A, Shamseer L, Weinstein E, Yazdi F, Turner L, Thielman J, et al. Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: systematic review. BMJ. 2014;348:g3804. https://doi.org/10.1136/bmj.g3804.View ArticlePubMedPubMed CentralGoogle Scholar
  20. Hirst A, Altman DG. Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals. PLoS One. 2012;7(4):e35621. https://doi.org/10.1371/journal.pone.0035621.View ArticlePubMedPubMed CentralGoogle Scholar
  21. Shamseer L, Hopewell S, Altman DG, Moher D, Schulz KF. Update on the endorsement of CONSORT by high impact factor journals: a survey of journal “Instructions to Authors” in 2014. Trials. 2016;17:301. https://doi.org/10.1186/s13063-016-1408-z.View ArticlePubMedPubMed CentralGoogle Scholar
  22. Goodman SN, Fanelli D, Ioannidis JP. What does research reproducibility mean? Sci Transl Med. 2016;8(341):341ps12. https://doi.org/10.1126/scitranslmed.aaf5027.View ArticlePubMedGoogle Scholar
  23. Couzin-Frankel J. Journals under the microscope. Science. 2018;361(6408):1180–3. https://doi.org/10.1126/science.361.6408.1180.

Copyright

© The Author(s). 2018

Advertisement