- Research article
- Open Access
Reporting of harms in oncological clinical study reports submitted to the European Medicines Agency compared to trial registries and publications—a methodological review
BMC Medicine volume 19, Article number: 88 (2021)
An accurate and comprehensive assessment of harms is a fundamental part of an accurate weighing of benefits and harms of an intervention when making treatment decisions; however, harms are known to be underreported in journal publications. Therefore, we sought to compare the completeness of reporting of harm data, discrepancies in harm data reported, and the delay to access results of oncological clinical trials between three sources: clinical study reports (CSRs), clinical trial registries and journal publications.
We used the EMA clinical data website to identify all trials submitted to the EMA between 2015 and 2018. We retrieved all CSRs and included all phase II, II/III or III randomised controlled trials (RCTs) assessing targeted therapy and immunotherapy for cancer. We then identified related records in clinical trial registries and journals. We extracted harms data for eight pre-specified variables and determined the completeness of reporting of harm data in each of the three sources.
We identified 42 RCTs evaluating 13 different drugs. Results were available on the EMA website in CSRs for 37 (88%) RCTs, ClinicalTrials.gov for 36 (86%), the European Clinical Trials Register (EUCTR) for 20 (48%) and in journal publications for 32 (76%). Harms reporting was more complete in CSRs than other sources. We identified marked discrepancies in harms data between sources, e.g. the number of patients discontinuing due to adverse events differed in CSRs and clinical trial registers for 88% of trials with data in both sources. For CSRs and publications, the corresponding number was 90%. The median (interquartile range) delay between the primary trial completion date and access to results was 4.34 (3.09–7.22) years for CSRs, 2.94 (1.16–4.52) years for ClinicalTrials.gov, 5.39 (4.18–7.33) years for EUCTR and 2.15 (0.64–5.04) years for publications.
Harms of recently approved oncological drugs were reported more frequently and in more detail in CSRs than in trial registries and journal publications. Systematic reviews seeking to address harms of oncological treatments should ideally use CSRs as the primary source of data; however, due to problems with access, this is currently not feasible.
Systematic reviews of randomised controlled trials (RCTs) are considered the gold standard for evaluating the effectiveness and harms of interventions . However, results of many completed RCTs are not published, which leads to reduced power and potential publication bias in reviews [2,3,4]. Moreover, peer-reviewed publications are not always an accurate reflection of how trials were planned, conducted and analysed. A lack of transparency or missing information on harms is common [3, 5].
One potential source of unpublished data is clinical study reports (CSRs): extensive reports prepared by pharmaceutical companies and submitted to regulatory authorities as a part of an application for marketing authorisation. The structure of CSRs is outlined in a guideline from the International Conference on Harmonisations . Access to CSRs has historically been difficult , but since 2015, the European Medicines Agency (EMA) has launched an initiative (policy 0700) to increase transparency of information on medicinal drugs by providing access to CSRs submitted to the agency. However, the EMA has not published any CSRs since December 4, 2018, when the initiative was paused indefinitely during the EMA’s move to Amsterdam . Although several systematic reviews have included CSRs [9,10,11,12] and a questionnaire study found that respondents consider CSRs valuable for systematic reviews , a 2014 study found that most systematic reviews continue to rely on publications as the primary source of data .
Several studies have compared reporting in publications, trial registries and CSRs; for example, a study found that CSRs had higher reporting quality than did registry reports and publications , a finding that was confirmed in several other studies [16,17,18,19,20]. However, no study has systematically compared reporting of harms in trial registries and publications with a large sample of recent CSRs from oncological trials.
Targeted therapy and immunotherapy for cancer have revolutionised the care of most patients with cancer. Several of these specific oncologic drugs have recently been approved by the US Food and Drug Administration and EMA. Evaluating the harms of these new drugs is essential. Thus, we aimed to compare the delay to access results of oncological RCTs, the completeness of reporting harm data and discrepancies in harm data reported between the three sources: CSRs available on the EMA Website, clinical trial registries and journal publications.
We identified all trials evaluating targeted therapy and immunotherapy for cancer available on the EMAs clinical data Website and retrieved the related CSRs. Then, we systematically searched for the related records in clinical trial registries and related publications. Finally, we compared the delay to access results, the completeness of reporting of harm data and discrepancies in harm data reported between the three sources.
Identification of trials
In November 2019, we used the EMA’s clinical data website (https://clinicaldata.ema.europa.eu) to identify all submissions for marketing authorisation or extension of indication under the EMA policy 0070. We updated the search in June 2020 and identified no new submissions. For all submissions, we extracted the product name, active substance, marketing authorisation holder and Anatomical Therapeutic Chemical (ATC) code. We selected the ATC codes for monoclonal antibodies (L01XC) and protein kinase inhibitors (L01XE) corresponding to targeted therapy and immunotherapy.
Once we had identified all eligible active substances, we downloaded all documents from the EMA website (i.e., CSRs and related documents) and used these to create a list of all trials submitted to the EMA. We included phase II, II/III or III RCTs that were part of a submission for a targeted therapy or immunotherapy. We excluded trials that compared only different dosages of the same treatment.
Two reviewers (ASP-M and PC) independently identified trials from the documents for one-quarter of the eligible active substances. Because of no discrepancies in this identification, the remaining identifications involved one reviewer (ASP-M).
Identification of related clinical-trial registry records for the identified RCTs
One reviewer (ASP-M) systematically searched ClinicalTrials.gov and the European Clinical Trials Register (EUCTR) by using (1) the trial registry identifier or ID number if mentioned in the CSR or (2) the name of the experimental drug (or its international non-proprietary name). If we were still unable to identify the corresponding trial, we used other keywords (e.g. treatment comparator and indication). The records identified were systematically checked by a second reviewer (PC). Then we checked whether results were posted on the trial registries identified. If the study was registered in both registries, we extracted data from both.
Identification of results publications for identified RCTs
We first searched for citations listed in trial registries. For ClinicalTrials.gov, the only registry to give access to citations, we used the citations listed under “publication of results” and “publications automatically indexed to this study by ClinicalTrials.gov identifier (NCT Number)”. We included all publications reporting results for the trial identified. We did not include publications of reviews or publications that presented pooled analyses of several trials and did not include data from the individual trial. If no publications were indexed in the registry record, we searched MEDLINE and EMBASE by using the name of the experimental drug, treatment comparator, indication and name of the principal investigator.
For each trial, we extracted information from the CSR available on the EMA Website, the clinical trial registry records (both ClinicalTrials.gov and EUCTR) and all related publications. The extracted information was entered in a data extraction spreadsheet. Two reviewers (ASP and PC) independently extracted data for 10% of trials. Because of only minor disagreements, one reviewer (ASP-M) extracted the data for the remaining trials. All extractions were then checked by a second reviewer (PC). All discrepancies were resolved by discussion.
We extracted the following information for each trial:
General characteristics: name of trial, name of studied drug, clinical development phase, condition; number of centres, number of arms, number of participants randomised, whether the trial was a non-inferiority trial, the primary outcome, funding and whether the trial was blinded.
Delay in access to trial results: The CSRs included in this project are released under the EMA policy 0070 , which dictates that clinical data submitted to the EMA as part of a marketing authorisation application or a post-authorisation procedure shall be released once the concerned procedure (hereafter EMA procedure) has been finalised. We recorded the date of finalisation of the procedure for all included submissions by using the European Commission’s register  and determined the delay between the finalisation of the procedure and publication of CSRs on the EMA website.
To determine the time between completion of the study and release of results in each source, we recorded the primary trial completion date (i.e. the date of the last participant’s final follow-up visit for measurement of the primary outcome) from ClinicalTrials.gov. If this was not available, we checked the other sources (CSRs, publications, and EUCTR) for a primary completion date. We also recorded for each source the date when the results were released and available. For trials with multiple publications, we used the earliest publication date. We then calculated the delay between primary trial completion date and availability of results for each source.
Completeness of reporting harm data and discrepancies in harm data: We extracted the following information from all three sources of data for each trial: number of patients randomised, whether a definition of safety population was provided, number of patients in the safety population, threshold for reporting adverse events (e.g. 10%, 5% or none), number of patients experiencing at least one adverse event, total number of adverse events, number of patients experiencing at least one serious adverse event, total number of serious adverse events, number of patients experiencing at least one adverse event judged to be grade 3–5 according to the Common Terminology Criteria for Adverse Events (CTCAE), total number of adverse events judged to be grade 3–5 according to the CTCAE, number of patients discontinuing the trial due to adverse events, number of deaths due to adverse events, and whether a description of the process of determining whether a death was due to adverse events, including whether the person(s) making the judgement were blinded, was provided. For all variables, we recorded the numbers per arm. Some sources reported CTCAE grade 3–4 events rather than grade 3–5 events. If the number of grade 5 events was reported separately, we added the numbers. If the number of grade 5 events was not reported, we still gave the trial a “yes” for the question, extracted the number of grade 3–4 events and noted this.
We compared reporting of the different variables defined above in CSRs with that in clinical trial registries and publications, separately. We performed Kaplan-Meier analysis on the delay from primary trial completion date to the publication of the CSR, the first publication of results in trial registries, and a publication in a medical journal. If a trial had not been published in a source, we calculated the delay between the primary completion date and June 29, 2020 and considered the trial right censored. For numerical variables reported in at least two of the data sources, we examined whether the numbers reported were the same. For this analysis, we pooled results from ClinicalTrials.gov and EUCTR. If results were available from both registries, we used the data from ClinicalTrials.gov for the analysis of discrepancies.
No patients were involved in the planning or conduct of this study.
Selection of trials
We identified 142 submissions through the EMA clinical data Website. These submissions corresponded to 124 unique substances: 22 concerned oncology drugs and 13 of these corresponded to targeted therapy and immunotherapy (Additional file 1). For these 13 drugs included in the study, we identified 164 unique trials, of which 42 met our eligibility criteria (phase II, II/III or III RCTs). The inclusion process is shown in Fig. 1.
The drugs included and the number of trials for each drug are in Table 1. The median number of randomised patients in the included trials was 364 (range 142–666) (Table 2). The primary outcome was progression-free survival for 27 of the 42 (64%) included trials, overall survival for eight (19%) and both for three (7%). The remaining four trials (7%) had other primary outcomes. Additional characteristics of included studies are in the Additional file 1: Table S1. Publications matched to individual trials are presented in Additional file 1: Table S2 [23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112].
Availability of sources
The EMA’s Website had complete CSRs for 37 of the 42 (88%) included trials. For the remaining five (12%) trials, the EMA Website did not contain full CSRs and only documents such as summaries, pharmacokinetic data or periodic safety reports were available; the EMA Website did not explain why full CSRs are not available for these trials, but three trials were ongoing at the time of the application for marketing authorisation. Among the 42 included trials, trial results were posted on ClinicalTrials.gov for 36 (86%) and on the EUCTR for 20 (48%). We were able to identify publications with results for 32 of the 42 (76%) included trials (all included publications are in the Additional file 1). Trial results were available in the three sources for 25 (60%) trials and in two sources for 13 (31%) (i.e. CSR and clinical trial registry for six trials, in CSR and publication for three trials, in clinical trial registry and publications for four trials). Results were available in only one source for three (7%) trials (two in CSRs and one in a clinical trial registry) and one (2%) trial had no results available.
Delay in access to trial results
The median delay between finalisation of the EMA procedure and availability of CSRs was 1.21 years (range 0.91–1.78). The median (interquartile range) delay between primary completion data and result availability was 4.34 (3.09–7.22) years for CSRs, 2.94 (1.16–4.52) years for ClinicalTrials.gov, 5.39 (4.18–7.33) years for the EUCTR and 2.15 (0.64–5.04) years for publications. Figure 2 shows Kaplan-Meier curves for the delay between primary trial completion dates and publication of the different sources.
Reports of harms in each source
Table 3 shows the proportion of trials for which we could obtain data on our pre-specified variables from each of the four sources of data. For most variables, results were more frequently reported in CSRs than both trial registries and publications. The number of patients with at least one serious adverse event was reported for all trials in both CSRs and ClinicalTrials.gov and in 19/20 (95%) trials in the EUCTR but only 16/32 (50%) trials with publications. The number of patients with any adverse events was reported in all CSRs but was not available for any trials or registries because the number of patients with serious and non-serious adverse events are reported separately. The number of patients with any adverse events was available for only 13/32 (41%) trials with publications. The number of patients with CTCAE grade 3–5 events was available in 36/37 (97%) CSRs but only 14/32 (44%) publications. The CTCAE grade was not reported in either of the trial registries.
The total number of serious adverse events, any adverse events, and CTCAE grade 3–5 events was available in 9/37 (24%), 12/37 (32%) and 6/37 (16%) CSRs, respectively; 10/36 (28%), 10/36 (28%) and 0/36 (0%) records at ClinicalTrials.gov; and 17/20 (85%), 17/20 (85%) and 0/20 (0%) records at the EUCTR. For publications, only 1/32 (3%) reports gave the total number of serious adverse events. The number of total adverse events and grade 3–5 adverse events was not available in publications for any trial.
The number of deaths due to adverse events was available in CSRs for 34/37 (92%) trials, from ClinicalTrials.gov for no trials, from EUCTR for 15/20 trials (75%) and from publications for 12/32 (38%) trials. Only 10/37 (27%) trials in CSRs and no trials in other sources had a detailed explanation of how it was decided whether a death was due to an adverse event or progressive disease.
Discrepancies between sources
For trials for which results were available for a variable in a minimum of two sources of data, we compared the data and noted any discrepancies. The proportion of trials with discrepancies are in Table 4. Figure 3 shows discrepancies for each variable in each included trial.
We found marked discrepancies in harms data between CSRs, trial registry entries and publications. The number of patients who discontinued the treatment due to adverse events was particularly problematic, with discrepancies in 88% and 90% of trials for CSRs as compared with registries and publications, respectively. The number of deaths due to adverse events differed between CSRs and clinical trial registers in 92% of trials with data from both sources, the corresponding number was 40% for CSRs and publications. Similarly, the total number of serious adverse events differed between CSRs and clinical trial registers for all five trials whereas the numbers matched in the single trial that had data in both a CSR and a publication.
Our study shows that data on harms in RCTs evaluating targeted therapy and immunotherapy for cancer are reported more frequently and in more detail in CSRs than in registries and publications. However, reporting is not perfect. CSRs were missing for five (12%) trials. Three of these trials were ongoing at the time of submission of documents to the EMA which might explain the missing CSRs. The two remaining trials were completed before the submission and it is unclear why CSRs were missing for these trials. Furthermore, important data were incompletely reported; for example, the total number of serious adverse events and all adverse events was available in only 9/37 (24%) and 12/37 (32%) CSRs. Although data should be available at the date the EMA procedure is completed, we showed a median of 1.21 years between the finalisation of the procedure and publication of CSRs on the website. Additionally, the delay from primary completion of a trial until results were available was longer for CSRs than for other sources. We also demonstrated discrepancies in harms data between CSRs and other sources. One possible explanation for such discrepancies could be different selection criteria for which events to include in which reports, e.g., clinicaltrials.gov use a 5% threshold for non-serious adverse events and similar thresholds are also commonly used in journal publications. Routinely reporting all events, without thresholds, would both improve reporting and potentially solve some discrepancies.
Our results are consistent with other findings. In 2013, Wieseler et al. examined a sample of 86 trials with both a CSR and a publicly available source of data and found that serious adverse events, adverse events and withdrawals due to adverse events were more frequently reported in CSRs than another source . In 2014, Maund et al. found that for nine antidepressant trials, CSRs were a more reliable source of information on harms than were journal articles . In 2016, two reports described the reporting of harms of orlistat in CSRs and journal publications: both concluded that reporting of harms was more extensive in CSRs than in journal publications [18, 19]. In 2019, a study compared six CSRs for gabapentin and two for quetiapine with publications and found that in CSRs all adverse events were reported, whereas no publications reported all adverse events .
Our finding of non-publication of RCTs and poor reporting of harms in publications is also in accordance with previous findings . The CONSORT statement outlines items that should be reported in journal publications describing RCTs and is endorsed by 585 journals . However, the CONSORT statement only has one item addressing harms. An extension for harms exists and some of the items outlined in this extension are the number of participants discontinued due to harms, the frequency of all adverse events with separate information about the severity, and the number of both affected participants and the number of events . Unfortunately, to our knowledge, submission of the CONSORT harms extension is not mandated by any journal, and of the Lancet, New England Journal of Medicine, BMJ and JAMA, only the Lancet makes specific reference to the extension in their guidance to authors . Had the extension been endorsed by the journals included in this study, and thus been followed, publications would likely have fared much better.
The substantial delay between completion of trials and availability of CSRs is an important barrier to access to all data. Several teams have highlighted the need to access trial results through posting on trial registries. For example, a report from TranspariMED states that in a 2013 cohort of cancer drug trials, two thirds of trials had not posted results to ClinicalTrials.gov 3 years after completion . Interventions have been proposed to improve access to results  and similar strategies to increase access and reduce delays for CSRs should be developed.
Our study is the first to compare reporting of harms in CSRs released under EMA policy 0070 with publications and trial registries for oncological trials. The automatic release of the CSRs might have led to better reporting of harms in other sources of data, but this does not seem to be the case. Additionally, we systematically examined predefined variables in a relatively large sample of trials.
Our study has some limitations. First, we focused on oncology trials, and our findings might not be applicable to other fields of medicine. However, our results, together with results from previous studies, suggest that the reporting of harms is better in CSRs than trial registries and journal publications across all specialities. Second, we examined only two clinical trial registries, and more information might be available from other registries; however, ClinicalTrials.gov and EUCTR are two of the most-used registries, and information available elsewhere is unlikely to substantially alter our conclusions.
Our study has important implications for both research and practice. Our results suggest that any systematic review or other assessment of harms associated with oncological treatments would have to rely on CSRs for making the soundest conclusions. If such an assessment relies on data from only publications and trial registries, it will only be able to include a subset of the available data. This is problematic for several reasons, namely reduced power to detect differences between groups and the fact that the data reported in registries and publications might vary systematically from data not reported. Additionally, we have shown marked discrepancies in data reported in CSRs and other sources, especially for withdrawals due to adverse events; therefore, we consider results based on CSRs more reliable. However, while we believe that CSRs are currently the most reliable source of data on harms, using CSRs might currently not be feasible. First, even though CSRs contained more data than other sources, important information was still missing in a significant number of CSRs. Secondly, we identified a substantial delay between the completion of trials and the availability of a CSR. To solve the problem of missing data in CSRs, we suggest that regulatory authorities make stricter requirements to the quality of submitted CSRs. The issue of delay in access is complicated by the fact that the EMA does not release CSRs until the procedure for which they were submitted is completed; however, we identified substantial delay from completion of the procedure to availability of CSR. By decreasing this delay to the absolute minimum, the overall delay could be reduced substantially. Currently, the EMA are not releasing any CSRs due to the agency’s move to Amsterdam, and in a reply to an open letter from IQWIQ and Cochrane the agency would not commit to reinitiating publication of CSRs .
In addition, improving reporting of harms in journal publications would be valuable, as this is a very accessible source of information and as publications are often available earlier than CSRs. By actively enforcing the CONSORT statement and the extension for harms, journals could help improve the reporting of harms. Also, the CONSORT extension for harms, which was released in 2004, could be updated to better reflect new opportunities for data-sharing and new knowledge on reporting of harms.
In addition, current estimates of the harms of oncological treatments based on published data might not be accurate and not able to inform clinical practice. Because oncological treatments are generally toxic, the harms profile is an important piece of information for assessing the benefit/harm balance, and true informed consent is only possible if the estimate of harms is accurate.
We suggest that future studies comparing reporting of harms in different sources should focus on more detailed aspects of harms reporting, e.g. whether information on adverse events by System Organ Class and Preferred Term levels are available.
Harms in trials evaluating targeted therapy and immunotherapy for cancer are reported in more detail and more reliably in CSRs than in trial registries and journal publications. This finding confirms previous results and suggests that CSR would provide more complete data on harms of oncological treatments and ideally systematic reviewers should have access to CSRs for providing accurate evidence synthesis. Thus, we consider it vital that regulators start routinely releasing CSRs.
Availability of data and materials
The CSRs used for this study are available at the EMA’s clinical data Website. All data files and the code for the statistical analysis are available from the Open Science Framework database (link: https://osf.io/mabyu/?view_only=d58aea7810234c229e593549569bc42d).
Anatomic therapeutic chemical
Clinical study report
Common Terminology Criteria for Adverse Events
European Medicines Agency
European Clinical Trials Register
Randomised controlled trial
Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312:71–2.
Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA. 1990;263:1385–9.
Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PloS One. 2013;8:e66844.
Scherer RW, Meerpohl JJ, Pfeifer N, Schmucker C, Schwarzer G, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2018;11:MR000005.
Hodkinson A, Kirkham JJ, Tudur-Smith C, Gamble C. Reporting of harms data in RCTs: a systematic review of empirical assessments against the CONSORT harms extension. BMJ Open. 2013;3. https://doi.org/10.1136/bmjopen-2013-003436.
The International Council for Harmonisation of Technical Requirements for Human Use (ICH). Guideline E3 - Structure and Content of Clinical Study Reports. 1996.
Gøtzsche PC, Jørgensen AW. Opening up data at the European medicines agency. BMJ. 2011;342. https://doi.org/10.1136/bmj.d2686.
European Medicines Agency. Clinical data publication. European Medicines Agency. 2018. https://www.ema.europa.eu/en/human-regulatory/marketing-authorisation/clinical-data-publication. Accessed 29 Jun 2020.
Maund E, Guski LS, Gøtzsche PC. Considering benefits and harms of duloxetine for treatment of stress urinary incontinence: a meta-analysis of clinical study reports. CMAJ. 2017;189:E194–203.
Jefferson T, Jones M, Doshi P, Spencer EA, Onakpoya I, Heneghan CJ. Oseltamivir for influenza in adults and children: systematic review of clinical study reports and summary of regulatory comments. BMJ. 2014;348:g2545.
Rohner E, Grabik M, Tonia T, Jüni P, Pétavy F, Pignatti F, et al. Does access to clinical study reports from the European medicines agency reduce reporting biases? A systematic review and meta-analysis of randomized controlled trials on the effect of erythropoiesis-stimulating agents in cancer patients. Plos One. 2017;12:e0189309.
Sharma T, Guski LS, Freund N, Gøtzsche PC. Suicidality and aggression during antidepressant treatment: systematic review and meta-analyses based on clinical study reports. BMJ. 2016;352:i65.
Hodkinson A, Dietz KC, Lefebvre C, Golder S, Jones M, Doshi P, et al. The use of clinical study reports to enhance the quality of systematic reviews: a survey of systematic review authors. Syst Rev. 2018;7:117.
Golder S, Loke YK, Wright K, Sterrantino C. Most systematic reviews of adverse effects did not include unpublished data. J Clin Epidemiol. 2016;77:125–33.
Wieseler B, Kerekes MF, Vervoelgyi V, McGauran N, Kaiser T. Impact of document type on reporting quality of clinical drug trials: a comparison of registry reports, clinical study reports, and journal publications. BMJ. 2012;344. https://doi.org/10.1136/bmj.d8141.
Wieseler B, Wolfram N, McGauran N, Kerekes MF, Vervölgyi V, Kohlepp P, et al. Completeness of reporting of patient-relevant clinical trial outcomes: comparison of unpublished clinical study reports with publicly available data. Plos Med. 2013;10:e1001526.
Maund E, Tendal B, Hróbjartsson A, Jørgensen KJ, Lundh A, Schroll J, et al. Benefits and harms in clinical trials of duloxetine for treatment of major depressive disorder: comparison of clinical study reports, trial registries, and publications. BMJ. 2014;348:g3510.
Hodkinson A, Gamble C, Smith CT. Reporting of harms outcomes: a comparison of journal publications with unpublished clinical study reports of orlistat trials. Trials. 2016;17:207.
Schroll JB, Penninga EI, Gøtzsche PC. Assessment of adverse events in protocols, clinical study reports, and published papers of trials of orlistat: a document analysis. Plos Med. 2016;13:e1002101.
Mayo-Wilson E, Fusco N, Hong H, Li T, Canner JK, Dickersin K. Opportunities for selective reporting of harms in randomized clinical trials: selection criteria for non-systematic adverse events. Trials. 2019;20:553.
European Medicines Agency. European Medicines Agency policy on publication of clinical data for medicinal products for human use. 2019. https://www.ema.europa.eu/en/documents/other/european-medicines-agency-policy-publication-clinical-data-medicinal-products-human-use_en.pdf.
European Commission. Union register of medicinal products. https://ec.europa.eu/health/documents/community-register/html/index_en.htm#foot2. Accessed 3 Feb 2020.
Barlesi F, Garon EB, Kim D-W, Felip E, Han J-Y, Kim J-H, Ahn M-J, Fidler MJ, Gubens MA, de Castro G, Surmont V, Li Q, Deitz AC, Lubiniecki GM, Herbst RS. Health-related quality of life in KEYNOTE-010: a phase II/III study of pembrolizumab versus docetaxel in patients with previously treated advanced, programmed death ligand 1–expressing NSCLC. J Thor Oncol. 2019;14(5):793–801.
Basch EM, Scholz M, de Bono JS, Vogelzang N, de Souza P, Marx G, Vaishampayan U, George S, Schwarz JK, Antonarakis ES, O'Sullivan JM, Kalebasty AR, Chi KN, Dreicer R, Hutson TE, Dueck AC, Bennett AV, Dayan E, Mangeshkar M, Holland J, Weitzman AL, Scher HI. Cabozantinib versus mitoxantrone-prednisone in symptomatic metastatic castration-resistant prostate cancer: a randomized phase 3 trial with a primary pain endpoint. Eur Urol. 2019;75(6):929–37.
Batchelor TT, Mulholland P, Neyns N, Nabors LB, Campone M, Wick A, Mason W, Mikkelsen T, Phuphanich S, Ashby LS, DeGroot J, Gattamaneni R, Cher L, Rosenthal M, Payer F, Jürgensmeier JM, Jain RK, Sorensen AG, Xu J, Liu Q, van den Bent M. Phase III randomized trial comparing the efficacy of cediranib as monotherapy, and in combination with lomustine, versus lomustine alone in patients with recurrent glioblastoma. J Clin Oncol. 2013;31(26):3212–8.
Bhadhuri A, Insinga R, Guggisberg P, Panje C, Schwenkglenks M. Cost effectiveness of pembrolizumab vs chemotherapy as firstline treatment for metastatic NSCLC that expresses high levels of PD-L1 in Switzerland. Swiss Med Wkly. 2019;149:w20170.
Borghaei H, Paz-Ares L, Horn L, Spigel DR, Steins M, Ready NE, Chow LQ, Vokes EE, Felip E, Holgado E, Barlesi F, Kohlhäufl M, Arrieta O, Burgio MA, Fayette J, Lena H, Poddubskaya E, Gerber DE, Gettinger SN, Rudin CM, Rizvi N, Crinò L, Blumenschein GR, Antonia SJ, Dorange C, Harbison CT, Finckenstein FG, Brahmer JR. Nivolumab versus Docetaxel in Advanced Nonsquamous Non–Small-Cell Lung Cancer. New Engl J Med. 2015;373(17):1627–39.
Brahmer JR, Rodríguez-Abreu D, Robinson AG, Hui R, Csőszi T, Fülöp A, Gottfried M, Peled N, Tafreshi A, Cuffe S, O'Brien M, Rao S, Hotta K, Zhang J, Lubiniecki GM, Deitz AC, Rangwala R, Reck M. Health-related quality-of-life results for pembrolizumab versus chemotherapy in advanced, PD-L1-positive NSCLC (KEYNOTE-024): a multicentre, international, randomised, open-label phase 3 trial. Lancet Oncol. 2017;18(12):1600–9.
Buzzoni R, Carnaghi C, Strosberg J, Fazio N, Singh S, Herbst F, Ridolfi A, Pavel M, Wolin E, Valle J, Oh D-Y, Yao J, Pommier R. Impact of prior therapies on everolimus activity: an exploratory analysis of RADIANT-4. OncoTargets Ther. 2017;10:5013–30.
Cappuzzo F, Ciuleanu T, Stelmakh L, Cicenas S, Szczésna A, Juhász E, Esteban E, Molinier O, Brugger W, Melezínek I, Klingelschmitt G, Klughammer B, Giaccone G. Erlotinib as maintenance treatment in advanced non-small-cell lung cancer: a multicentre, randomised, placebo-controlled phase 3 study. Lancet Oncol. 2010;11(6):521–9.
Carlino MS, Long GV, Schadendorf D, Robert C, Ribas A, Richtig E, Nyakas M, Caglevic C, Tarhini A, Blank C, Hoeller C, Bar-Sela G, Barrow C, Wolter P, Zhou H, Emancipator K, Jensen EH, Ebbinghaus S, Ibrahim N, Daud A, Outcomes by line of therapy and programmed death ligand 1 expression in patients with advanced melanoma treated with pembrolizumab or ipilimumab in KEYNOTE-006: a randomised clinical trial. Eur J Cancer. 2018;101:236–43.
Cella D, Escudier B, Tannir NM, Powles T, Donskov F, Peltola K, Schmidinger M, Heng DYC, Mainwaring PN, Hammers HJ, Lee JL, Roth BJ, Marteau F, Williams P, Baer J, Mangeshkar M, Scheffold C, Hutson TE, Pal S, Motzer RJ, Choueiri TK. Quality of life outcomes for cabozantinib versus everolimus in patients with metastatic renal cell carcinoma: METEOR phase III randomized trial. J Clin Oncol. 2018;36(8):757–64.
Cella D, Grünwald V, Nathan P, Doan J, Dastani H, Taylor F, Bennett B, DeRosa M, Berry S, Broglio K, Berghorn E, Motzer RJ. Quality of life in patients with advanced renal cell carcinoma given nivolumab versus everolimus in CheckMate 025: a randomised, open-label, phase 3 trial. Lancet Oncol. 2016;17(7):994–1003.
Choueiri TK, Escudier B, Powles T, Tannir NM, Mainwaring PN, Rini BI, Hammers HJ, Donskov F, Roth BJ, Peltola K, Lee JL, Heng DYC, Schmidinger M, Agarwal N, Sternberg CN, McDermott DF, Aftab DT, Hessel C, Scheffold C, Schwab G, Hutson TE, Pal S, Motzer RJ, Cabozantinib versus everolimus in advanced renal cell carcinoma (METEOR): final results from a randomised, open-label, phase 3 trial. Lancet Oncol. 2016;17(7):917–27.
Choueiri TK, Escudier B, Powles T, Mainwaring PN, Rini BI, Donskov F, Hammers H, Hutson TE, Lee J-L, Peltola K, Roth BJ, Bjarnason GA, Géczi L, Keam B, Maroto P, Heng DYC, Schmidinger M, Kantoff PW, Borgman-Hagey A, Hessel C, Scheffold C, Schwab GM, Tannir NM, Motzer RJ. Cabozantinib versus everolimus in advanced renal-cell carcinoma. N Engl J Med. 2015;373(19):1814–23.
Ciuleanu T, Tsai C-M, Tsao C-J, Milanowski J, Amoroso D, Heo DS, Groen HJM, Szczesna A, Chung C-Y, Chao T-Y, Middleton G, Zeaiter A, Klingelschmitt G, Klughammer B, Thatcher N. A phase II study of erlotinib in combination with bevacizumab versus chemotherapy plus bevacizumab in the first-line treatment of advanced non-squamous non-small cell lung cancer. Lung Cancer. 2013;82(2):276–81.
Clement PM, Gauler T, Machiels JP, Haddad RI, Fayette J, Licitra LF, Tahara M, Cohen EEW, Cupissol D, Grau JJ, Guigay J, Caponigro F, de Castro G, de Souza Viana L, Keilholz U, del Campo JM, Cong XJ, Ehrnrooth E, Vermorken JB. Afatinib versus methotrexate in older patients with secondline recurrent and/or metastatic head and neck squamous cell carcinoma: subgroup analysis of the LUX-Head & Neck 1 trial. Ann Oncol. 2016;27(8):1585–93.
Cohen EEW, Licitra LF, Burtness B, Fayette J, Gauler T, Clement PM, Grau JJ, del Campo JM, Mailliez A, Haddad RI, Vermorken JB, Tahara M, Guigay J, Geoffrois L, Merlano MC, Dupuis N, Krämer N, Cong XJ, Gibson N, Solca F, Ehrnrooth E, Machiels J-PH. Biomarkers predict enhanced clinical outcomes with afatinib versus methotrexate in patients with second-line recurrent and/or metastatic head and neck cancer. Ann Oncol. 2017;28 (10):2526–32.
Costa C, Molina MA, Drozdowskyj A, Giménez-Capitán A, Bertran-Alamillo J, Karachaliou N, Gervais R, Massuti B, Wei J, Moran T, Majem M, Felip E, Carcereny E, Garcia-Campelo R, Viteri S, Taron M, Ono M, Giannikopoulos P, Bivona T, Rosell R. The impact of T790M Mutations and mRNA expression on outcome in patients with -mutant NSCLC treated with erlotinib or chemotherapy in the randomized phase III EURTAC trial . Clin Cancer Res. 2014;20(7):2001–10.
Cristofanilli M, DeMichele A, Giorgetti C, Turner NC, Slamon DJ, Im S-A, Masuda N, Verma S, Loi S, Colleoni M, Theall KP, Huang X, Liu Y, Bartlett CH. Predictors of prolonged benefit from palbociclib plus fulvestrant in women with endocrine-resistant hormone receptor–positive/human epidermal growth factor receptor 2–negative metastatic breast cancer in PALOMA-3. Eur J Cancer. 2018;104:21–31.
Dueck AC, Scher HI, Bennett AV, Mazza GL, Thanarajasingam G, Schwab G, Weitzman AL, Rogak LJ, Basch E. Assessment of adverse events from the patient perspective in a phase 3 metastatic castration-resistant prostate cancer clinical trial. JAMA Oncol. 2020;6(2):e193332.
Elisei R, Schlumberger MJ, Müller SP, Schöffski P, Brose MS, Shah MH, Licitra L, Jarzab B, Medvedev V, Kreissl MC, Niederle B, Cohen EEW, Wirth LJ, Ali H, Hessel C, Yaron Y, Ball D, Nelkin B, Sherman SI. Cabozantinib in progressive medullary thyroid cancer. J Clin Oncol. 2013;31(29):3639–46.
Escudier B, Motzer RJ, Sharma P, Wagstaff J, Plimack ER, Hammers HJ, Donskov F, Gurney H, Sosman JA, Zalewski PG, Harmenberg U, McDermott DF, Choueiri TK, Richardet M, Tomita Y, Ravaud A, Doan J, Zhao H, Hardy H, George S. Treatment beyond progression in patients with advanced renal cell carcinoma treated with nivolumab in CheckMate 025. Eur Urol. 2017;72(3):368–76.
Escudier B, Powles T, Motzer RJ, Olencki T, Frontera oA, Oudard S, Rolland F, Tomczak P, Castellano D, Appleman LJ, Drabkin H, Vaena D, Milwee S, Youkstetter J, Lougheed JC, Bracarda S, Choueiri TK. Cabozantinib, a new standard of care for patients with advanced renal cell carcinoma and bone metastases? Subgroup analysis of the METEOR Trial. J Clin Oncol. 2018;36(8):765–72.
Escudier B, Sharma P, McDermott DF, George S, Hammers HJ, Srinivas S, Tykodi SS, Sosman JA, Procopio G, Plimack ER, Castellano D, Gurney H, Donskov F, Peltola K, Wagstaff J, Gauler TC, Ueda T, Zhao H, Waxman IM, Motzer RJ. CheckMate 025 randomized phase 3 study: outcomes by key baseline factors and prior therapy for nivolumab versus everolimus in advanced renal cell carcinoma. Eur Urol. 2017;72(6):962–71.
Evans TRJ, Kudo M, Finn RS, Han K-H, Cheng A-L, Ikeda M, Kraljevic S, Ren M, Dutcus CE, Piscaglia F, Sung MW. Urine protein:creatinine ratio vs 24-hour urine protein for proteinuria management: analysis from the phase 3 REFLECT study of lenvatinib vs sorafenib in hepatocellular carcinoma. Br J Cancer. 2019;121(3):218–21.
Fazio N, Buzzoni R, Fave GD, Tesselaar ME, Wolin E, Van Cutsem E, Tomassetti P, Strosberg J, Voi M, Bubuteishvili-Pacaud L, Ridolfi A, Herbst F, Tomasek J, Singh S, Pavel M, Kulke MH, Valle JW, Yao JC. Everolimus in advanced, progressive, well-differentiated, non-functional neuroendocrine tumors: RADIANT-4 lung subgroup analysis. Cancer Sci. 2018;109(1):174–81.
Finn RS, Martin M, Rugo HS, Jones S, Im S-A, Gelmon K, Harbeck N, Lipatov ON, Walshe JM, Moulder S, Gauthier E, Lu DR, Randolph S, Diéras V, Slamon DJ. Palbociclib and letrozole in advanced breast cancer. N Engl J Med. 2016;375(20):1925–36.
Gadgeel S, Goss G, Soria J-C, Felip E, Georgoulias V, Lu S, Cobo M, Syrigos K, Lee KH, Göker E, Guclu SZ, Isla D, Morabito A, Dupuis N, Bühnemann C, Krämer N, Solca F, Ehrnrooth E, Ardizzoni A. Evaluation of the VeriStrat® serum protein test in patients with advanced squamous cell carcinoma of the lung treated with second-line afatinib or erlotinib in the phase III LUX-Lung 8 study. Lung Cancer. 2017;109:101–8.
Goss GD, Arnold A, Shepherd FA, Dediu M, Ciuleanu T-E, Fenton D, Zukin M, Walde D, Laberge F, Vincent MD, Ellis PM, Laurie SA, Ding K, Frymire E, Gauthier I, Leighl NB, Ho C, Noble J, Lee CW, Seymour L. Randomized, double-blind trial of carboplatin and paclitaxel with either daily oral cediranib or placebo in advanced non–small-cell lung cancer: NCIC Clinical Trials Group BR24 Study. J Clin Oncol. 2010;28(1):49–55.
Goss GD, Felip E, Cobo M, Lu S, Syrigos K, Lee KH, Göker E, Georgoulias V, Li W, Guclu S, Isla D, Min YJ, Morabito A, Ardizzoni A, Gadgeel SM, Fülöp A, Bühnemann C, Gibson N, Krämer N, Solca F, Cseh A, Ehrnrooth E, Soria J-C. Association of mutations with clinical outcomes of afatinib- or erlotinib-treated patients with lung squamous cell carcinoma . JAMA Oncol. 2018;4(9):1189.
Harbeck N, Iyer S, Turner N, Cristofanilli M, Ro J, André F, Loi S, Verma S, Iwata H, Bhattacharyya H, Puyana Theall K, Bartlett CH, Loibl S. Quality of life with palbociclib plus fulvestrant in previously treated hormone receptor-positive, HER2-negative metastatic breast cancer: patient-reported outcomes from the PALOMA-3 trial. Ann Oncol. 2016;27(6):1047–54.
Herbst RS, Baas P, Perez-Gracia JL, Felip E, Kim D-W, Han J-Y, Molina JR, Kim J-H, Dubos Arvis C, Ahn M-J, Majem M, Fidler MJ, Surmont V, de Castro G, Garrido M, Shentu Y, Emancipator K, Samkari A, Jensen EH, Lubiniecki GM, Garon EB. Use of archival versus newly collected tumor samples for assessing PD-L1 expression and overall survival: an updated analysis of KEYNOTE-010 trial. Ann Oncol. 2019;30(2):281–9.
Herbst RS, Ansari R, Bustin F, Flynn P, Hart L, Otterson GA, Vlahovic G, Soh C-H, O'Connor P, Hainsworth J. Efficacy of bevacizumab plus erlotinib versus erlotinib alone in advanced non-small-cell lung cancer after failure of standard first-line chemotherapy (BeTa): a double-blind, placebo-controlled, phase 3 trial. Lancet. 2011;377(9780):1846–54.
Herbst RS, Baas P, Kim D-W, Felip E, Pérez-Gracia JL, Han J-Y, Molina J, Kim J-H, Arvis CD, Ahn M-J, Majem M, Fidler MJ, de Castro G, Garrido M, Lubiniecki GM, Shentu Y, Im E, Dolled-Filhart M, Garon EB. Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial. Lancet. 2016;387(10027):1540–50.
Hodi FS, Chesney J, Pavlick AC, Robert C, Grossmann KF, McDermott DF, Linette GP, Meyer N, Giguere JK, Agarwala SS, Shaheen M, Ernstoff MS, Minor DR, Salama AK, Taylor MH, Ott PA, Horak C, Gagnier P, Jiang J, Wolchok JD, Postow MA. Combined nivolumab and ipilimumab versus ipilimumab alone in patients with advanced melanoma: 2-year overall survival outcomes in a multicentre, randomised, controlled, phase 2 trial. Lancet Oncol. 2016;17(11):1558–68.
Hodi FS, Chiarion-Sileni V, Gonzalez R, Grob J-J, Rutkowski P, Cowey CL, Lao CD, Schadendorf D, Wagstaff J, Dummer R, Ferrucci PF, Smylie M, Hill A, Hogg D, Marquez-Rodas I, Jiang J, Rizzo J, Larkin J, Wolchok JD. Nivolumab plus ipilimumab or nivolumab alone versus ipilimumab alone in advanced melanoma (CheckMate 067): 4-year outcomes of a multicentre, randomised, phase 3 trial. Lancet Oncol. 2018;19(11):1480–92.
Karachaliou N, Gimenez-Capitan A, Drozdowskyj A, Viteri S, Moran T, Carcereny E, Massuti B, Vergnenegre A, de Marinis F, Molina MA, Teixido C, Rosell R. ROR1 as a novel therapeutic target for EGFR-mutant non-small-cell lung cancer patients with the EGFR T790M mutation. Transl Lung Cancer Res. 2014;3(3).
Karachaliou N, Mayo-de las Casas C, Queralt C, de Aguirre I, Melloni B, Cardenal F, Garcia-Gomez R, Massuti B, Sánchez JM, Porta R, Ponce-Aix S, Moran T, Carcereny E, Felip E, Bover I, Insa A, Reguart N, Isla D, Vergnenegre A, de Marinis F, Gervais R, Corre R, Paz-Ares L, Morales-Espinosa D, Viteri S, Drozdowskyj A, Jordana-Ariza N, Ramirez-Serrano JL, Molina-Vila MA, Rosell R. Association of L858R Mutation in Circulating Free DNA With Survival in the EURTAC Trial . JAMA Oncol. 2015;1(2):149.
Kiyota N, Schlumberger M, Muro K, Ando Y, Takahashi S, Kawai Y, Wirth L, Robinson B, Sherman S, Suzuki T, Fujino K, Gupta A, Hayato S, Tahara M. Subgroup analysis of Japanese patients in a phase 3 study of lenvatinib in radioiodinerefractory differentiated thyroid cancer. Cancer Sci. 2015;106(12):1714–21.
Kudo M, Finn RS, Qin S, Han K-H, Ikeda K, Piscaglia F, Baron A, Park J-W, Han G, Jassem J, Blanc JF, Vogel A, Komov D, Evans TRJ, Lopez C, Dutcus C, Guo M, Saito K, Kraljevic S, Tamai T, Ren M, Cheng A-L. Lenvatinib versus sorafenib in first-line treatment of patients with unresectable hepatocellular carcinoma: a randomised phase 3 noninferiority trial. Lancet. 2018;391(10126):1163–73.
Larkin J, Chiarion-Sileni V, Gonzalez R, Grob JJ, Cowey CL, Lao CD, Schadendorf D, Dummer R, Smylie M, Rutkowski P, Ferrucci PF, Hill A, Wagstaff J, Carlino MS, Haanen JB, Maio M, Marquez-Rodas I, McArthur GA, Ascierto PA, Long GV, Callahan MK, Postow MA, Grossmann K, Sznol M, Dreno B, Bastholt L, Yang A, Rollin LM, Horak C, Hodi FS, Wolchok JD. Combined Nivolumab and Ipilimumab or Monotherapy in Untreated Melanoma. N Engl J Med. 2015;373(1):23–34.
Larkin J, Chiarion-Sileni V, Gonzalez R, Grob J-J, Rutkowski P, Lao CD, Cowey CL, Schadendorf D, Wagstaff J, Dummer R, Ferrucci PF, Smylie M, Hogg D, Hill A, Márquez-Rodas I, Haanen J, Guidoboni M, Maio M, Schöffski P, Carlino MS, Lebbé C, McArthur G, Ascierto PA, Daniels GA, Long GV, Bastholt L, Rizzo JI, Balogh A, Moshyk A, Hodi FS, Wolchok JD. Five-year survival with combined nivolumab and ipilimumab in advanced melanoma. N Engl J Med. 2019;381(16):1535–46.
Laurie SA, Solomon BJ, Seymour L, Ellis PM, Goss GD, Shepherd FA, Boyer MJ, Arnold AM, Clingan P, Laberge F, Fenton D, Hirsh V, Zukin M, Stockler MR, Lee CW, Chen EX, Montenegro A, Ding K, Bradbury PA. Randomised, double-blind trial of carboplatin and paclitaxel with daily oral cediranib or placebo in patients with advanced non-small cell lung cancer: NCIC Clinical Trials Group study BR29. Eur J Cancer. 2014;50(4):706–12.
Ledermann JA, Embleton AC, Raja F, Perren TJ, Jayson GC, Rustin GJS, Kaye SB, Hirte H, Eisenhauer E, Vaughan M, Friedlander M, González-Martín A, Stark D, Clark E, Farrelly L, Swart AM, Cook A, Kaplan RS, Parmar MKB. Cediranib in patients with relapsed platinum-sensitive ovarian cancer (ICON6): a randomised, double-blind, placebo controlled phase 3 trial. Lancet. 2016;387(10023):1066–74.
Loibl S, Turner NC, Ro J, Cristofanilli M, Iwata H, Im S-A, Masuda N, Loi S, André F, Harbeck N, Verma S, Folkerd E, Theall KP, Hoffman J, Zhang K, Bartlett CH, Dowsett M. Palbociclib combined with fulvestrant in premenopausal women with advanced breast cncer and prior progression on endocrine therapy: PALOMA‐3 results. Oncologist. 2017;22(9):1028–38.
Lu S, Li W, Zhou C, Hu C-P, Qin S, Cheng G, Feng J, Wang J, Cseh A, Peil B, Gibson N, Ehrnrooth E, Zhang L. Afatinib vs erlotinib for second-line treatment of Chinese patients with advanced squamous cell carcinoma of the lung. OncoTargets Ther. 2018;11:8565–73.
Machiels J-PH, Haddad RI, Fayette J, Licitra LF, Tahara M, Vermorken JB, Clement PM, Gauler T, Cupissol D, Grau JJ, Guigay J, Caponigro F, de Castro G, de Souza Viana L, Keilholz U, del Campo JM, Cong XJ, Ehrnrooth E, Cohen EEW. Afatinib versus methotrexate as second-line treatment in patients with recurrent or metastatic squamous-cell carcinoma of the head and neck progressing on or after platinum-based therapy (LUX-Head & Neck 1): an open-label, randomised phase 3 trial. Lancet Oncol. 2015;16(5):583–94.
Masuda N, Inoue K, Nakamura R, Rai Y, Mukai H, Ohno S, Hara F, Mori Y, Hashigaki S, Muramatsu Y, Nagasawa T, Umeyama Y, Huang X, Iwata H. Palbociclib in combination with fulvestrant in patients with hormone receptorpositive, human epidermal growth factor receptor 2-negative advanced breast cancer: PALOMA-3 subgroup analysis of Japanese patients. Int J Clin Oncol. 2019;24(3):262–73.
McGuire WP, Penson RT, Gore M, Herraez AC, Peterson P, Shahir A, Ilaria R. Randomized phase II study of the PDGFRα antibody olaratumab plus liposomal doxorubicin versus liposomal doxorubicin alone in patients with platinum-refractory or platinum-resistant advanced ovarian cancer. BMC Cancer. 2018;18(1).
Miller VA, Hirsh V, Cadranel J, Chen Y-M, Park K, Kim S-W, Zhou C, Su W-C, Wang M, Sun Y, Heo DS, Crino L, Tan E-H, Chao T-Y, Shahidi M, Cong XJ, Lorence RM, Yang JC-H. Afatinib versus placebo for patients with advanced, metastatic non-small-cell lung cancer after failure of erlotinib, gefitinib, or both, and one or two lines of chemotherapy (LUX-Lung 1): a phase 2b/3 randomised trial. Lancet Oncol. 2012;13(5):528–38.
Motzer RJ, Escudier B, McDermott DF, George S, Hammers HJ, Srinivas S, Tykodi SS, Sosman JA, Procopio G, Plimack ER, Castellano D, Choueiri TK, Gurney H, Donskov F, Bono P, Wagstaff J, Gauler TC, Ueda T, Tomita Y, Schutz FA, Kollmannsberger C, Larkin J, Ravaud A, Simon JS, Xu L-A, Waxman IM, Sharma P. Nivolumab versus everolimus in avanced renal-cell carcinoma. N Engl J Med. 2015;373(19):1803–13.
Mukai H, Shimizu C, Masuda N, Ohtani S, Ohno S, Takahashi M, Yamamoto Y, Nishimura R, Sato N, Ohsumi S, Iwata H, Mori Y, Hashigaki S, Muramatsu Y, Nagasawa T, Umeyama Y, Lu DR, Toi M. Palbociclib in combination with letrozole in patients with estrogen receptor–positive, human epidermal growth factor receptor 2–negative advanced breast cancer: PALOMA-2 subgroup analysis of Japanese patients. Int J Clin Oncol. 2019;24(3):274–87.
Mulders P, Hawkins R, Nathan P, de Jong I, Osanto S, Porfiri E, Protheroe A, van Herpen CML, Mookerjee B, Pike L, Jürgensmeier JM, Gore ME. Cediranib monotherapy in patients with advanced renal cell carcinoma: results of a randomised phase II study. Eur J Cancer. 2012;48(4):527–37.
Pavel ME, Singh S, Strosberg JR, Bubuteishvili-Pacaud L, Degtyarev E, Neary MP, Carnaghi C, Tomasek J, Wolin E, Raderer M, Lahner H, Valle JW, Pommier R, Van Cutsem E, Tesselaar MET, Fave GD, Buzzoni R, Hunger M, Eriksson J, Cella D, Ricci J-F, Fazio N, Kulke MH, Yao JC. Health-related quality of life for everolimus versus placebo in patients with advanced, non-functional, well-differentiated gastrointestinal or lung neuroendocrine tumours (RADIANT-4): a multicentre, randomised, double-blind, placebo-controlled, phase 3 trial. Lancet Oncol. 2017;18 (10):1411–22.
Petrella TM, Robert C, Richtig E, Miller WH, Masucci GV, Walpole E, Lebbe C, Steven N, Middleton MR, Hille D, Zhou W, Ibrahim N, Cebon J. Patient-reported outcomes in KEYNOTE-006, a randomised study of pembrolizumab versus ipilimumab in patients with advanced melanoma. Eur J Cancer. 2017;86:115–24.
Postow MA, Chesney J, Pavlick AC, Robert C, Grossmann K, McDermott D, Linette GP, Meyer N, Giguere JK, Agarwala SS, Shaheen M, Ernstoff MS, Minor D, Salama AK, Taylor M, Ott PA, Rollin LM, Horak C, Gagnier P, Wolchok JD, Hodi FS. Nivolumab and Ipilimumab versus Ipilimumab in Untreated Melanoma. N Engl J Med. 2015;372(21):2006–17.
Reck M, Brahmer J, Bennett B, Taylor F, Penrod JR, DeRosa M, Dastani H, Spigel DR, Gralla RJ. Evaluation of health-related quality of life and symptoms in patients with advanced non-squamous non-small cell lung cancer treated with nivolumab or docetaxel in CheckMate 057. Eur J Cancer. 2018;102:23–30
Reck M, Rodríguez-Abreu D, Robinson AG, Hui R, Csőszi T, Fülöp A, Gottfried M, Peled N, Tafreshi A, Cuffe S, O’Brien M, Rao S, Hotta K, Leiby MA, Lubiniecki GM, Shentu Y, Rangwala R, Brahmer JR. Pembrolizumab versus chemotherapy for PD-L1–positive non–small-cell lung cancer. N Engl J Med. 2016;375(19):1823–33.
Robert C, Ribas A, Schachter J, Arance A, Grob J-J, Mortier L, Daud A, Carlino MS, McNeil CM, Lotem M, Larkin JMG, Lorigan P, Neyns B, Blank CU, Petrella TM, Hamid O, Su S-C, Krepler C, Ibrahim N, Long GV. Pembrolizumab versus ipilimumab in advanced melanoma (KEYNOTE-006): post-hoc 5-year results from an open-label, multicentre, randomised, controlled, phase 3 study. Lancet Oncol. 2019;20(9):1239–51.
Robert C, Schachter J, Long GV, Arance A, Grob JJ, Mortier L, Daud A, Carlino MS, McNeil C, Lotem M, Larkin J, Lorigan P, Neyns B, Blank CU, Hamid O, Mateus C, Shapira-Frommer R, Kosh M, Zhou H, Ibrahim N, Ebbinghaus S, Ribas A. Pembrolizumab versus ipilimumab in advanced melanoma. N Engl J Med. 2015;372(26):2521–32.
Robertson JD, Botwood NA, Rothenberg ML, Schmoll H-J. Phase III trial of FOLFOX plus bevacizumab or cediranib (AZD2171) as first-line treatment of patients with metastatic colorectal cancer: HORIZON III. Clin Colorectal Cancer. 2009;8(1):59–60.
Robinson B, Schlumberger M, Wirth LJ, Dutcus CE, Song J, Taylor MH, Kim S-B, Krzyzanowska MK, Capdevila J, Sherman SI, Tahara M. Characterization of tumor size changes over time from the phase 3 study of lenvatinib in thyroid cancer. J Clin Endocrinol Metab. 2016;101(11):4103–9.
Rosell R, Carcereny E, Gervais R, Vergnenegre A, Massuti B, Felip E, Palmero R, Garcia-Gomez R, Pallares C, Sanchez JM, Porta R, Cobo M, Garrido P, Longo F, Moran T, Insa A, De Marinis F, Corre R, Bover I, Illiano A, Dansin E, de Castro J, Milella M, Reguart N, Altavilla G, Jimenez U, Provencio M, Moreno MA, Terrasa J, Muñoz-Langa J, Valdivia J, Isla D, Domine M, Molinier O, Mazieres J, Baize N, Garcia-Campelo R, Robinet G, Rodriguez-Abreu D, Lopez-Vivanco G, Gebbia V, Ferrera-Delgado L, Bombaron P, Bernabe R, Bearz A, Artal A, Cortesi E, Rolfo C, Sanchez-Ronco M, Drozdowskyj A, Queralt C, de Aguirre I, Ramirez JL, Sanchez JJ, Molina MA, Taron M, Paz-Ares L. Erlotinib versus standard chemotherapy as first-line treatment for European patients with advanced EGFR mutation-positive non-small-cell lung cancer (EURTAC): a multicentre, open-label, randomised phase 3 trial. Lancet Oncol. 2012;13(3):239–46.
Rugo HS, Diéras V, Gelmon KA, Finn RS, Slamon DJ, Martin M, Neven P, Shparyk Y, Mori A, Lu DR, Bhattacharyya H, Bartlett CHUANG, Iyer S, Johnston S, Ettl J, Harbeck N. Impact of palbociclib plus letrozole on patient-reported health-related quality of life: results from the PALOMA-2 trial. Ann Oncol. 2018;29(4):888–94.
Rugo HS, Finn RS, Gelmon K, Joy AA, Harbeck N, Castrellon A, Mukai H, Walshe JM, Mori A, Gauthier E, Lu DR, Bananis E, Martin M, Diéras V. Progression-free survival outcome is independent of objective response in patients with estrogen receptor-positive, human epidermal growth factor receptor 2-negative advanced breast cancer treated with palbociclib plus letrozole compared with letrozole: analysis from PALOMA-2. Clin Breast Cancer. 2020;20(2):e173–80.
Schachter J, Ribas A, Long GV, Arance A, Grob J-J, Mortier L, Daud A, Carlino MS, McNeil C, Lotem M, Larkin J, Lorigan P, Neyns B, Blank C, Petrella TM, Hamid O, Zhou H, Ebbinghaus S, Ibrahim N, Robert C. Pembrolizumab versus ipilimumab for advanced melanoma: final overall survival results of a multicentre, randomised, open-label phase 3 study (KEYNOTE-006). Lancet. 2017;390(10105):1853–62.
Schadendorf D, Larkin J, Wolchok J, Hodi FS, Chiarion-Sileni V, Gonzalez R, Rutkowski P, Grob J-J, Cowey CL, Lao C, Wagstaff J, Callahan MK, Postow MA, Smylie M, Ferrucci PF, Dummer R, Hill A, Taylor F, Sabater J, Walker D, Kotapati S, Abernethy A, Long GV. Health-related quality of life results from the phase III CheckMate 067 study. Eur J Cancer. 2017;82:80–91.
Schlumberger M, Elisei R, Müller S, Schöffski P, Brose M, Shah M, Licitra L, Krajewska J, Kreissl MC, Niederle B, Cohen EEW, Wirth L, Ali H, Clary DO, Yaron Y, Mangeshkar M, Ball D, Nelkin B, Sherman S. Overall survival analysis of EXAM, a phase III trial of cabozantinib in patients with radiographically progressive medullary thyroid carcinoma. Ann Oncol. 2017;28(11):2813–9.
Schlumberger M, Tahara M, Wirth LJ, Robinson B, Brose MS, Elisei R, Habra MA, Newbold K, Shah MH, Hoff AO, Gianoukakis AG, Kiyota N, Taylor MH, Kim S-B, Krzyzanowska MK, Dutcus CE, de las Heras B, Zhu J, Sherman SI. Lenvatinib versus placebo in radioiodine-refractory thyroid cancer. N Engl J Med. 2015;372(7):621–30.
Schuler M, Yang JC-H, Park K, Kim J-H, Bennouna J, Chen Y-M, Chouaid C, De Marinis F, Feng J-F, Grossi F, Kim D-W, Liu X, Lu S, Strausz J, Vinnyk Y, Wiewrodt R, Zhou C, Wang B, Chand VK, Planchard D, Ou SI, Planchard D, Park K, Schuler M, Yang J, Chand V, Rohr K, Bagnes C, Martin CM, Recondo G, Zarba JJ, Blajman C, Richardet M, McLachlan S-A, Parente P, Underhill C, Crombie C, Mainwaring P, Greil R, Humblet Y, Bustin F, Carestia L, Galdermans D, Lambrechts M, Delval L, Vercauter P, Zhou C, Wang J, Huang C, Lin X, Wu Y, Liu X, Cheng Y, Qin S, Feng J, Huang J, Zhang Y, Lu S, Zereu M, Garicochea B, Zadra CA, Riska H, Alanko T, Cadranel J, Chouaid C, Zalcman G, Sibilot DM, Perol M, Planchard D, Bennouna J, Fournel P, Gervais R, Rotarski M, Coudert B, Schuler M, Thomas M, Wehler T, Faehling M, Keilholz U, Laack E, von Pawel J, Huber R, Dickgreber N, Wiewrodt R, Mark Z, Tehenes S, Strausz J, Sarosi V, Prabhash K, Jain M, Venkatesan S, Sharma L, Dadhich H, Nagarkar RV, Onn A, Gottfried M, Stemmer S, Migliorino MR, Grossi F, Bidoli P, Bearz A, Gridelli C, Milandri C, Platania M, Ceresoli GL, Cruciani G, Delgado FG, Perez JLG, Luna GA, Baca OP, Aerts JGJV, Stigt JA, Dingemans AMC, Herder GJM, Gans SJM, Sánchez JFS, Alvarez Barreda RL, Pantigoso WR, Palomino OLM, Jaskiewicz P, Kazarnowicz A, Serwatowski P, Szczesna A, Jassem J, Lubennikov V, Karaseva N, Orlov S, Ragulin Y, Garrido P, Larriba JLG, Camps C, Campelo RG, Lianes P, Cobo M, Felip E, Kim D-W, Kim S-W, Park K, Kim J-H, Han J-Y, Kim Y-C, Yang C-H, Hsia T-C, Chen Y-M, Tsai Y-H, Chang G-C, Tsao TC-Y, Su W-C, Huang M-S, Ho C-L, Hsieh R-K, Vinnyk Y, Popovych O, Ponomarova O, Bondarenko I, Polishchuk I, Shah R, Mitra S, Popat S, Spicer J, Toy E, Popat S, Talbot T, Brown E, Upadhyay S, Summers Y, Gurtler J, Meza L, Thropay J. Afatinib beyond progression in patients with non-small-cell lung cancer following chemotherapy, erlotinib/gefitinib and afatinib: phase III randomized LUX-Lung 5 trial. Ann Oncol. 2016;27(3):417–23.
Schuler M, Wu Y-L, Hirsh V, O’Byrne K, Yamamoto N, Mok T, Popat S, Sequist LV, Massey D, Zazulina V, Yang JC-H. First-line afatinib versus chemotherapy in patients with non–small cell lung cancer and common epidermal growth factor receptor gene mutations and brain metastases. J Thor Oncol. 2016;11(3):380–90.
Shah R, Botteman M, Solem CT, Luo L, Doan J, Cella D, Motzer RJ. A Quality-adjusted time without symptoms or toxicity (Q-TWiST) analysis of nivolumab versus everolimus in advanced renal cell carcinoma (aRCC). Clin Genitourin Cancer. 2019;17(5):356–65.e1.
Smith JC, Brooks L, Hoff PM, McWalter G, Dearden S, Morgan SR, Wilson D, Robertson JD, Jürgensmeier JM. KRAS mutations are associated with inferior clinical outcome in patients with metastatic colorectal cancer, but are not predictive for benefit with cediranib. Eur J Cancer. 2013;49(10):2424–32.
Smith M, De Bono J, Sternberg C, Le Moulec S, Oudard S, De Giorgi U, Krainer M, Bergman A, Hoelzer W, De Wit R, Bögemann M, Saad F, Cruciani G, Thiery-Vuillemin A, Feyerabend S, Miller K, Houédé N, Hussain S, Lam E, Polikoff J, Stenzl A, Mainwaring P, Ramies D, Hessel C, Weitzman A, Fizazi K. Phase III study of cabozantinib in previously treated metastatic castration-resistant prostate cancer: COMET-1. J Clin Oncol. 2016;34(25):3005–13.
Soria J-C, Felip E, Cobo M, Lu S, Syrigos K, Lee KH, Göker E, Georgoulias V, Li W, Isla D, Guclu SZ, Morabito A, Min YJ, Ardizzoni A, Gadgeel SM, Wang B, Chand VK, Goss GD. Afatinib versus erlotinib as second-line treatment of patients with advanced squamous cell carcinoma of the lung (LUX-Lung 8): an open-label randomised controlled phase 3 trial. Lancet Oncol. 2015;16(8):897–907.
Tahara M, Brose MS, Wirth LJ, Suzuki T, Miyagishi H, Fujino K, Dutcus CE, Gianoukakis A. Impact of dose interruption on the efficacy of lenvatinib in a phase 3 study in patients with radioiodine-refractory differentiated thyroid cancer. Eur J Cancer. 2019;106:61–8.
Tahara M, Schlumberger M, Elisei R, Habra MA, Kiyota N, Paschke R, Dutcus CE, Hihara T, McGrath S, Matijevic M, Kadowaki T, Funahashi Y, Sherman SI. Exploratory analysis of biomarkers associated with clinical outcomes from the study of lenvatinib in differentiated cancer of the thyroid. Eur J Cancer. 2017;75:213–21.
Turner NC, Finn RS, Martin M, Im S-A, DeMichele A, Ettl J, Diéras V, Moulder S, Lipatov O, Colleoni M, Cristofanilli M, Lu DR, Mori A, Giorgetti C, Iyer S, Huang Bartlett C, Gelmon KA. Clinical considerations of the role of palbociclib in the management of advanced breast cancer patients with and without visceral metastases. Ann Oncol. 2018;29(3):669–80.
Turner NC, Liu Y, Zhu Z, Loi S, Colleoni M, Loibl S, DeMichele A, Harbeck N, André F, Bayar MA, Michiels S, Zhang Z, Giorgetti C, Arnedos M, Bartlett CH, Cristofanilli M. Cyclin E1 Expression and palbociclib efficacy in previously treated hormone receptor–positive metastatic breast cancer. J Clin Oncol. 2019;37(14):1169–78.
Turner NC, Slamon DJ, Ro J, Bondarenko I, Im S-A, Masuda N, Colleoni M, DeMichele A, Loi S, Verma S, Iwata H, Harbeck N, Loibl S, André F, Theall KP, Huang X, Giorgetti C, Bartlett CH, Cristofanilli M. Overall survival with palbociclib and fulvestrant in advanced breast cancer. N Engl J Med. 2018;379(20):1926–36.
van Vugt MJH, Stone JA, “Rik” De Greef HJMM, Snyder ES, Lipka L, Turner DC, Chain A, Lala M, Li M, Robey SH, Kondic AG, De Alwis D, Mayawala K, Jain L, Freshwater T. Immunogenicity of pembrolizumab in patients with advanced tumors. J ImmunoTher Cancer. 2019;7(1).
Verma S, Bartlett CH, Schnell P, DeMichele AM, Loi S, Ro J, Colleoni M, Iwata H, Harbeck N, Cristofanilli M, Zhang K, Thiele A, Turner NC, Rugo HS. Palbociclib in combination with fulvestrant in women with hormone receptor‐positive/HER2‐negative advanced metastatic breast cancer: detailed safety analysis from a multicenter, randomized, placebo‐controlled, phase III study (PALOMA‐3). Oncologist. 2016;21(10):1165–75.
Wolchok JD, Chiarion-Sileni V, Gonzalez R, Rutkowski P, Grob J-J, Cowey CL, Lao CD, Wagstaff J, Schadendorf D, Ferrucci PF, Smylie M, Dummer R, Hill A, Hogg D, Haanen J, Carlino MS, Bechter O, Maio M, Marquez-Rodas I, Guidoboni M, McArthur G, Lebbé C, Ascierto PA, Long GV, Cebon J, Sosman J, Postow MA, Callahan MK, Walker D, Rollin L, Bhore R, Hodi FS, Larkin J. Overall survival with combined nivolumab and ipilimumab in advanced melanoma. N Engl J Med. 2017;377(14):1345–56.
Wu Y-L, Sequist LV, Tan E-H, Geater SL, Orlov S, Zhang L, Lee KH, Tsai C-M, Kato T, Barrios CH, Schuler M, Hirsh V, Yamamoto N, O’Byrne K, Boyer M, Mok T, Peil B, Märten A, Yang JC-H, Paz-Ares L, Park K. Afatinib as First-line Treatment of Older Patients With EGFR Mutation-Positive Non-Small-Cell Lung Cancer: Subgroup analyses of the LUX-Lung 3, LUX-Lung 6, and LUX-Lung 7 trials. Clin Lung Cancer. 2018;19(4):e465–79.
Wu Y-L, Xu C-R, Hu C-P, Feng J, Lu S, Huang Y, Li W, Hou M, Shi JH, Märten A, Fan J, Peil B, Zhou C. 2018;11:8575–87.
Wu Y-L, Zhou C, Hu C-P, Feng J, Lu S, Huang Y, Li W, Hou M, Shi JH, Lee KY, Xu C-R, Massey D, Kim M, Shi Y, Geater SL. Afatinib versus cisplatin plus gemcitabine for first-line treatment of Asian patients with advanced nonsmall-cell lung cancer harbouring EGFR mutations (LUX-Lung 6): an open-label, randomised phase 3 trial. Lancet Oncol. 2014;15(2):213–22.
Yamashita T, Kudo M, Ikeda K, Izumi N, Tateishi R, Ikeda M, Aikata H, Kawaguchi Y, Wada Y, Numata K, Inaba Y, Kuromatsu R, Kobayashi M, Okusaka T, Tamai T, Kitamura C, Saito K, Haruna K, Okita K, Kumada H. REFLECT—a phase 3 trial comparing efficacy and safety of lenvatinib to sorafenib for the treatment of unresectable hepatocellular carcinoma: an analysis of Japanese subset. J Gastroenterol. 2020;55(1):113–22.
Yang JC-H, Sequist LV, Zhou C, Schuler M, Geater SL, Mok T, Hu C-P, Yamamoto N, Feng J, O'Byrne K, Lu S, Hirsh V, Huang Y, Sebastian M, Okamoto I, Dickgreber N, Shah R, Märten A, Massey D, Wind S, Wu Y-L. Effect of dose adjustment on the safety and efficacy of afatinib for EGFR mutation-positive lung adenocarcinoma: post hoc analyses of the randomized LUX-Lung 3 and 6 trials. Ann Oncol. 2016;27(11):2103–10.
Yang JC-H, Sequist LV, Geater SL, Tsai C-M, Mok TSK, Schuler M, Yamamoto N, Yu C-J, Ou S-HI, Zhou C, Massey D, Zazulina V, Wu Y-L. Clinical activity of afatinib in patients with advanced non-small-cell lung cancer harbouring uncommon EGFR mutations: a combined post-hoc analysis of LUX-Lung 2, LUX-Lung 3, and LUX-Lung 6. Lancet Oncol. 2015;16(7):830–8.
Yang JC-H, Wu Y-L, Schuler M, Sebastian M, Popat S, Yamamoto N, Zhou C, Hu C-P, O'Byrne K, Feng J, Lu S, Huang Y, Geater SL, Lee KY, Tsai C-M, Gorbunova V, Hirsh V, Bennouna J, Orlov S, Mok T, Boyer M, Su W-C, Lee KH, Kato T, Massey D, Shahidi M, Zazulina V, Sequist LV. Afatinib versus cisplatin-based chemotherapy for EGFR mutation-positive lung adenocarcinoma (LUX-Lung 3 and LUX-Lung 6): analysis of overall survival data from two randomised, phase 3 trials. Lancet Oncol. 2015;16(2):141–51.
Yao JC, Fazio N, Singh S, Buzzoni R, Carnaghi C, Wolin E, Tomasek J, Raderer M, Lahner H, Voi M, Pacaud LB, Rouyrre N, Sachs C, Valle JW, Fave GD, Van Cutsem E, Tesselaar M, Shimada Y, Oh D-Y, Strosberg J, Kulke MH, Pavel ME. Everolimus for the treatment of advanced, non-functional neuroendocrine tumours of the lung or gastrointestinal tract (RADIANT-4): a randomised, placebo-controlled, phase 3 study. Lancet. 2016;387(10022):968–77.
Golder S, Loke YK, Wright K, Norman G. Reporting of adverse events in published and unpublished studies of health care interventions: a systematic review. Plos Med. 2016;13. https://doi.org/10.1371/journal.pmed.1002127.
Schulz KF, Altman DG, Moher D. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340. https://doi.org/10.1136/bmj.c332.
Ioannidis JPA, Evans SJW, Gøtzsche PC, O’Neill RT, Altman DG, Schulz K, et al. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med. 2004;141:781–8.
Phillips R, Hazell L, Sauzet O, Cornelius V. Analysis and reporting of adverse events in randomised controlled trials: a review. BMJ Open. 2019;9:e024537.
Transparency International, TranspariMED. CLINICAL TRIAL TRANSPARENCY - A guide for policy makers 2017. https://www.transparency.org.uk/sites/default/files/pdf/publications/ClinicalTrials_FINAL_2.pdf.
Maruani A, Boutron I, Baron G, Ravaud P. Impact of sending email reminders of the legal requirement for posting results on ClinicalTrials.gov: cohort embedded pragmatic randomized controlled trial. BMJ. 2014;349:g5579.
European Medicines Agency. Response to the Institute for Quality and Efficiency in Health Care (IQWiG) and the Cochrane Collaboration on transparency of COVID-19-related activities. 2020. https://www.ema.europa.eu/en/documents/other/european-medicines-agency-response-iqwig-transparency-covid-19-related-activities_en.pdf.
We thank the European Medicines Agency (EMA) for making available the CSRs for this study; Cherifa Cheurfa for her previous work, which served as inspiration for this project; and Klaus Munkholm for helpful comments and suggestions on how to present our findings.
This study received no specific funding.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Paludan-Müller, A.S., Créquit, P. & Boutron, I. Reporting of harms in oncological clinical study reports submitted to the European Medicines Agency compared to trial registries and publications—a methodological review. BMC Med 19, 88 (2021). https://doi.org/10.1186/s12916-021-01955-0
- Systematic reviews
- Reporting bias
- Clinical study reports
- Adverse events