Reporting studies on time to diagnosis: proposal of a guideline by an international panel (REST)
- Elise Launay1, 2Email author,
- Jérémie F. Cohen1, 3,
- Patrick M. Bossuyt4,
- Pierre Buekens5,
- Jonathan Deeks6,
- Timothy Dye7,
- Richard Feltbower8,
- Andrea Ferrari9,
- Michael Kramer10,
- Mariska Leeflang4,
- David Moher11,
- Karel G. Moons12,
- Erik von Elm13,
- Philippe Ravaud14 and
- Martin Chalumeau1, 3View ORCID ID profile
© The Author(s). 2016
Received: 12 July 2016
Accepted: 8 September 2016
Published: 27 September 2016
Studies on time to diagnosis are an increasing field of clinical research that may help to plan corrective actions and identify inequities in access to healthcare. Specific features of time to diagnosis studies, such as how participants were selected and how time to diagnosis was defined and measured, are poorly reported. The present study aims to derive a reporting guideline for studies on time to diagnosis.
Each item of a list previously used to evaluate the completeness of reporting of studies on time to diagnosis was independently evaluated by a core panel of international experts (n = 11) for relevance and readability before an open electronic discussion allowed consensus to be reached on a refined list. The list was then submitted with an explanatory document to first, last and/or corresponding authors (n = 98) of published systematic reviews on time to diagnosis (n = 45) for relevance and readability, and finally approved by the core expert panel.
The refined reporting guideline consists of a 19-item checklist: six items are about the process of participant selection (with a suggested flowchart), six about the definition and measurement of time to diagnosis, and three about optional analyses of associations between time to diagnosis and participant characteristics and health outcomes. Of 24 responding authors of systematic reviews, more than 21 (≥88 %) rated the items as relevant, and more than 17 (≥70 %) as readable; 19 of 22 (86 %) authors stated that they would potentially use the reporting guideline in the future.
We propose a reporting guideline (REST) that could help authors, reviewers, and editors of time to diagnosis study reports to improve the completeness and the accuracy of their reporting.
KeywordsTime to diagnosis Reporting guideline Risk of bias Generalizability Research methodology
Time to diagnosis (TTD) is the interval from first alert symptoms to the diagnosis of a disease in a patient [1–3]. Studies on TTD may aim at (1) measuring the length of TTD and its evolution over time, (2) identifying the determinants of long TTD (i.e., relationship between TTD and patient or healthcare system characteristics), and/or (3) evaluating relationships between long versus short TTD and patient outcomes [1, 4]. The number of published studies on TTD is rapidly increasing . TTD is also of increasing concern among patient advocacy groups, and some, such as the International Confederation of Childhood Cancer Parent Organizations, have established a reduction in TTD as a priority . There are strong and common beliefs that longer TTD is associated with worse health outcomes, as revealed by various systematic reviews of the literature [1–3]. Furthermore, in PubMed, the definition of the Medical Subject Heading “early diagnosis” includes the following statement: ‘Generally, early diagnosis improves prognosis and treatment outcome’ . However, many primary studies and systematic reviews do not report worse patient outcomes as a consequence of longer versus shorter TTD, or have found inverse relationships between longer TTD and better patient outcomes [1, 2]. These paradoxical associations are due to incomplete adjustment and residual confounding, as observed in studies of TTD of pediatric brain and bone tumors [2, 7, 8]. Another issue associated with “early diagnosis” is related to the risk of over-diagnosis and over-treatment .
Studies on TTD are most commonly based on series of diagnosed cases and therefore have specific design features exposing them to risk of bias and threats to the generalizability of their findings . For example, bias may be introduced by how participants were selected, how TTD was defined and measured, and how the association with participant characteristics and health outcomes was assessed [1, 2, 4]. In a systematic review evaluating the quality of reporting of 50 studies on TTD, we found that these critical methodological aspects were rarely reported. Hence, critical appraisal of these studies by authors of systematic reviews was found frequently inadequate .
In a previous work, we showed that the reporting of primary studies on TTD was poor, notably for their specific design features. We therefore hypothesized that available reporting guidelines (e.g., STROBE , STARD , and Aarhus statement ) could be insufficient for accurate and complete reporting of studies on TTD [1, 4]. A first checklist was previously developed by some of us to evaluate the completeness of reporting of studies on TTD in the field of pediatrics . The aim of the present study was to derive a Reporting guideline for studies on TTD, using this previous checklist and the expertise of two independent international panels .
First version of the checklist
A preliminary checklist of 25 items was submitted to the scientific committee. This checklist included 23 (of 35) items generated in previous work in which we had evaluated the quality of reporting of studies on TTD in the field of pediatrics (Additional file 1) . These 23 items had been considered potentially related to risk of bias and threats to generalizability . The 12 other items were generic (e.g., aim of the study and inclusion criteria) common with other related reporting guidelines (e.g., STROBE, STARD); they were not initially submitted to the scientific committee but were reintegrated during the discussion process (Fig. 1). Two items from the Aarhus statement concerning TTD definition and measurement were added .
Discussion and refinement step
We identified 16 potential members for the scientific committee, comprising clinical research methodologists, clinical epidemiologists, clinicians, specialists in the field of TTD studies, and medical editors. We invited them to participate in an online survey by rating the relevance and readability of each item. We asked them to answer “yes” or “no” to the two following questions: “Is this item relevant?” and “Is this item readable?” Experts from the scientific committee were also invited to provide comments and to suggest additional items. After the first round of rating, we collated and anonymized all answers and comments, and sent them back to members of the scientific committee, together with a revised version of the checklist. Members of the scientific committee were then invited to provide feedback on the revised checklist and to propose modifications through an online general discussion. Revisions were made until we obtained agreement from all members of the scientific committee. The revised checklist was then used in the second broad rating step.
Broad rating step
We invited the first author, last author and corresponding (if different) author of each systematic review on TTD identified in a previous methodological review  to participate in an online survey in SurveyMonkey®. Authors from this broad panel were asked to rate the relevance and the readability of each item using the same questions as in the core refinement step. We added the possibility to answer “I don’t know” to the question on relevance. Experts could also leave comments on each item or the checklist and were asked if they would use this reporting guideline to report a study on TTD in the future. To facilitate understanding of items specific to studies on TTD, we provided a document with explanations and examples alongside the survey. Non-responders were sent two reminders. None of the 11 experts in the scientific committee participated in the broad rating step.
Finalization of the checklist and the explanatory document
Members of the scientific committee were invited to discuss deletion from the checklist for items with less than 50 % of agreement on relevance in the broad rating step, and to consider rephrasing for items with less than 50 % of agreement on readability. Comments of the broad rating panel were also used to complete the explanatory document. A final version of the reporting guideline, comprising a checklist and an explanatory document, was approved by the scientific committee through online discussions.
Discussion and refinement step
Broad rating step
Among 98 authors of 45 systematic reviews on TTD who were invited to participate, 29 (30 %) answered at least one question and 24 (24 %) rated all the 19 items. The topics of the systematic reviews written (or co-written) by the 98 authors were similar to those of the 29 responding authors (Additional file 3). Seventeen (59 %) of the 29 responding authors had already published a primary study on TTD, in addition to one or more systematic reviews.
All items were rated as relevant by at least 21 authors (88 % of the 24 responding authors). All items were rated as readable by at least 17 (71 %) of the 24 responding authors who rated all the items, while 19 items were rated as readable by at least 19 (80 % of the 24 responding authors) (Additional file 4). Among the 22 authors who answered the question, 19 (86 %) stated that they would potentially use the reporting guideline in the future. Among these, 10 had already conducted one or more primary studies on TTD. Three authors of systematic reviews commented that the Aarhus statement would be more appropriate to their fields of research.
Final version of the checklist
Given that the 19 proposed items were deemed relevant and readable by a majority of respondents of the broad rating panel, we decided to submit an unchanged checklist to the scientific committee. The explanatory document was completed and also submitted to the scientific committee.
Checklist for reporting studies on time to diagnosis (REST). More detailed explanation on items and examples are given in Additional file 5
Identify the article as a study on time to diagnosis
Explain the scientific background and rationale for the study
State specific objective(s)
Describe the setting, location(s), and relevant dates, including periods of recruitment
State eligibility criteria of participants (i.e., inclusion and exclusion criteria, especially diagnostic criteria)
Describe the source population (i.e., the population with signs and symptoms that usually trigger healthcare professionals to initiate the diagnostic procedures) and how the participants were identified within it
State how known subgroups of participants with an inherent individual risk of short or long time to diagnosis were handled (e.g., by subgroup analysis, exclusion)
Define time points (e.g., time of first signs and symptoms, time of diagnosis) and time intervalsb (e.g., patient or physician intervals)
State the methods used to collect study data (e.g., extraction from medical records, participant interview or questionnaires, analysis of an already existing database, other)
Describe how time points were assessed (e.g., number of assessors, their qualifications)
If the study aimed to evaluate associations between participant characteristics and time to diagnosis, state whether assessors of time to diagnosis were blinded to these characteristics
If the study aimed to evaluate associations between time to diagnosis and participant health outcomes (e.g., survival), state whether assessors of time to diagnosis were blinded to these outcomes
Describe the statistical methods used, including whether time to diagnosis was analyzed as a continuous or categorized variable (e.g., delayed versus not delayed)
If the study aimed to evaluate associations between time to diagnosis and other factors (e.g., participant characteristics or health outcomes), describe which confounders were considered and how they were chosen, measured and analyzed
Give a rationale for the sample size
Report the number of individuals at each step of the selection process between the source population and participants with data analyzed and provide a flowchart (see example); give reasons for non-participation at each stage
Report demographic and clinical characteristics of participants
Report the distribution of time to diagnosis
If associations between time to diagnosis and other factors (e.g., participant characteristics or health outcomes) were described, report measures of association and their precision (e.g., confidence intervals)
Summarize key results with reference to study objectives and discuss their potential clinical implications
Discuss sources of potential bias, including bias due to the selection of participants from the source population (e.g., undiagnosed cases) and to the assessment of time points
If association between time to diagnosis and survival was studied, discuss possible lead-time bias
We propose a new guideline (REST) for TTD studies to help authors, reviewers, and editors improve the reporting of these studies. The 19-item checklist (Table 1), along with the explanatory document (Additional file 5), have been approved by an international panel of experts in the field of research methodology, epidemiology and TTD studies, journal editors, and also by 71 to 88 % of responding authors of systematic reviews on TTD published before 2014. Our reporting guideline highlights key aspects that are specific to studies on TTD: how participants were selected, the definition and measurement of TTD, and the analyses of the association between TTD and participant characteristics and health outcomes. The flowchart suggested to illustrate each step of the participant selection process could help raise awareness among authors, reviewers, and editors of threats to generalizability of study findings and conclusions. Indeed, the differences between the source and the analyzed populations were found to be an important source of concern in the abovementioned systematic review of studies on TTD .
A complete and accurate report of the definition and measurement of TTD is also of importance to enable readers to critically evaluate the risk that estimates of TTD may be biased by a subjective definition of time points (i.e., time of first symptoms and time of diagnosis). Whether the assessment of these time points was blinded to potential determinants and health outcomes is also critical.
Recommendations for the development of reporting guidelines suggest that key information concerning risk of bias should be included in all new or revised reporting guidelines. Ideally, identification of key information should rely on the empirical demonstration that it is related to risk of bias or threats to generalizability. For example, such demonstration has helped illustrate the influence of randomization methods on the estimation of a treatment effect in randomized controlled trials, supporting their emphasis in CONSORT . In the present reporting guideline, judgment on each item’s potential influence on results was not based on an empirical demonstration, but on expert consensus. At present, an empirical demonstration is not possible, owing to the limited number of TTD studies investigating the same disease.
The methods used to develop this checklist are almost all in accordance with recommendations from the EQUATOR network, with a preparatory literature analysis  and vigilance about theoretical risk of bias and threats to generalizability . For feasibility reasons, the discussion and refinement step was conducted by on-line discussions rather than a face-to-face meeting as recommended by the EQUATOR network. This on-line discussion could have limited the debate but it also allowed an independent expression of experts’ opinions without risk of authoritarian arguments. The response rate in the broad rating step was low (30 %), but the topics covered by the systematic review published by responding authors were representative of those of the invited authors. In the broad rating step, participants were all authors of systematic reviews; their methodological background is therefore likely to have been stronger than that of most potential users of our reporting guideline. An explanatory document was produced to help less initiated authors to properly use the reporting guideline.
By estimating the magnitude of TTD for a disease, evaluating its potential consequences on prognosis and understanding its determinants, reports of studies on TTD may be used by clinicians and decision makers to plan corrective actions and identify inequities in access to healthcare. Critical appraisal is a key step before using results of clinical research for decisions in healthcare and policy. Incomplete and inaccurate reporting may prevent adequate critical appraisal of a study and lead to non-contributory scientific research . Our reporting guideline provides authors, reviewers, and editors, as well as users of studies on TTD, with a tool that promotes complete and accurate reporting of studies.
Availability of data and materials
All data supporting the results presented are included within the article or its supporting information.
EL and MC had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. EL, JFC, PR, and MC designed the study. PR and MC supervised the study. All authors contributed to analysis and interpretation of the data. EL, JFC, and MC drafted the manuscript. All authors critically revised the manuscript for important intellectual content. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
This work does not involve any humans, human data, or animals.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Launay E, Morfouace M, Deneux-Tharaux C, Gras le-Guen C, Ravaud P, Chalumeau M. Quality of reporting of studies evaluating time to diagnosis: a systematic review in paediatrics. Arch Dis Child. 2014;99:244–50. doi:https://doi.org/10.1136/archdischild-2013-304778.View ArticlePubMedGoogle Scholar
- Brasme J-F, Morfouace M, Grill J, Martinot A, Amalberti R, Bons-Letouzey C, et al. Delays in diagnosis of paediatric cancers: a systematic review and comparison with expert testimony in lawsuits. Lancet Oncol. 2012;13:e445–59. doi:https://doi.org/10.1016/S1470-2045(12)70361-3.View ArticlePubMedGoogle Scholar
- Scherdel P, Dunkel L, van Dommelen P, Goulet O, Salaün J-F, Brauner R, et al. Growth monitoring as an early detection tool: a systematic review. Lancet Diabetes Endocrinol. 2016;4(5):447–56. doi:https://doi.org/10.1016/S2213-8587(15)00392-7.View ArticlePubMedGoogle Scholar
- Launay E, Cohen JF, Morfouace M, Gras-Le Guen C, Ravaud P, Chalumeau M. Inadequate critical appraisal of studies in systematic reviews of time to diagnosis. J Clin Epidemiol. 2016. Ahead of print. doi:https://doi.org/10.1016/j.jclinepi.2016.03.013.
- International Confederation of Childhood Cancer Parent Organizations. 2013. http://cms.onlinebase.nl/userfiles/c1icccpo/file/WHO_leaflet.pdf. Accessed 20 Sept 2016.
- National Library of Medicine. Early diagnosis. Medical Subject Headings. National Library of Medicine, Bethesda, USA. http://www.ncbi.nlm.nih.gov/mesh/?term=early+diagnosis. Accessed 28 Jun 2015.
- Brasme J-F, Chalumeau M, Oberlin O, Valteau-Couanet D, Gaspar N. Time to diagnosis of Ewing tumors in children and adolescents is not associated with metastasis or survival: a prospective multicenter study of 436 patients. J Clin Oncol. 2014;32:1935–40. doi:https://doi.org/10.1200/JCO.2013.53.8058.View ArticlePubMedGoogle Scholar
- Brasme J-F, Grill J, Doz F, Lacour B, Valteau-Couanet D, Gaillard S, et al. Long time to diagnosis of medulloblastoma in children is not associated with decreased survival or with worse neurological outcome. PLoS One. 2012;7:e33415. doi:https://doi.org/10.1371/journal.pone.0033415.View ArticlePubMedPubMed CentralGoogle Scholar
- Moynihan R, Henry D, Moons KGM. Using evidence to combat overdiagnosis and overtreatment: evaluating treatments, tests, and disease definitions in the time of too much. PLoS Med. 2014;11:e1001655. doi:https://doi.org/10.1371/journal.pmed.1001655.View ArticlePubMedPubMed CentralGoogle Scholar
- von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61:344–9. doi:https://doi.org/10.1016/j.jclinepi.2007.11.008.View ArticleGoogle Scholar
- Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527.View ArticlePubMedPubMed CentralGoogle Scholar
- Weller D, Vedsted P, Rubin G, Walter FM, Emery J, Scott S, et al. The Aarhus statement: improving design and reporting of studies on early cancer diagnosis. Br J Cancer. 2012;106:1262–7. doi:https://doi.org/10.1038/bjc.2012.68.View ArticlePubMedPubMed CentralGoogle Scholar
- Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7:e1000217. doi:https://doi.org/10.1371/journal.pmed.1000217.View ArticlePubMedPubMed CentralGoogle Scholar
- Development of a reporting guideline for reporting studies on time to diagnosis. http://www.equator-network.org/wp-content/uploads/2009/02/Reporting-studies-on-time-to-diagnosis-summary.pdf. Accessed 30 Aug 2015.
- Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408–12.View ArticlePubMedGoogle Scholar
- Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383:267–76. doi:https://doi.org/10.1016/S0140-6736(13)62228-X.View ArticlePubMedGoogle Scholar