Skip to main content
Download PDF

Risk and benefit for umbrella trials in oncology: a systematic review and meta-analysis

BMC Medicine volume 20, Article number: 219 (2022) Cite this article

Abstract

Background

Umbrella clinical trials in precision oncology are designed to tailor therapies to the specific genetic changes within a tumor. Little is known about the risk/benefit ratio for umbrella clinical trials. The aim of our systematic review with meta-analysis was to evaluate the efficacy and safety profiles in cancer umbrella trials testing targeted drugs or a combination of targeted therapy with chemotherapy.

Methods

Our study was prospectively registered in PROSPERO (CRD42020171494). We searched Embase and PubMed for cancer umbrella trials testing targeted agents or a combination of targeted therapies with chemotherapy. We included solid tumor studies published between 1 January 2006 and 7 October 2019. We measured the risk using drug-related grade 3 or higher adverse events (AEs), and the benefit by objective response rate (ORR), progression-free survival (PFS), and overall survival (OS). When possible, data were meta-analyzed.

Results

Of the 6207 records identified, we included 31 sub-trials or arms of nine umbrella trials (N = 1637). The pooled overall ORR was 17.7% (95% confidence interval [CI] 9.5–25.9). The ORR for targeted therapies in the experimental arms was significantly lower than the ORR for a combination of targeted therapy drugs with chemotherapy: 13.3% vs 39.0%; p = 0.005. The median PFS was 2.4 months (95% CI 1.9–2.9), and the median OS was 7.1 months (95% CI 6.1–8.4). The overall drug-related death rate (drug-related grade 5 AEs rate) was 0.8% (95% CI 0.3–1.4), and the average drug-related grade 3/4 AE rate per person was 0.45 (95% CI 0.40–0.50).

Conclusions

Our findings suggest that, on average, one in five cancer patients in umbrella trials published between 1 January 2006 and 7 October 2019 responded to a given therapy, while one in 125 died due to drug toxicity. Our findings do not support the expectation of increased patient benefit in cancer umbrella trials. Further studies should investigate whether umbrella trial design and the precision oncology approach improve patient outcomes.

Peer Review reports

Background

Precision oncology is a strategy aiming to divide cancer patients into groups that will most likely respond to a given therapy. Treatment is tailored to the molecular makeup of a tumor rather than the site or stage of disease [1].

Umbrella trials are novel trial designs commonly used in precision oncology [2, 3] defined as trials with “many different treatment arms within one trial. People are assigned to a particular treatment arm of the trial based on their type of cancer and the specific molecular makeup of their cancer” [4].

The umbrella design is a type of master protocol which allows for testing multiple agents simultaneously and may include specified modifications while the trial is ongoing [5,6,7,8,9,10]. For example, adaptive randomization is often used to assign patients to the most effective experimental treatment based on continuous data accumulation and interim analyses [11]. These features of umbrella design are considered as having the potential to accelerate the process of drug development and maximize the benefits for trial participants [12]. Umbrella trials may be also classified as platform trials [7, 13] when participants are adaptively randomized and the protocol permits considerable flexibility to add new arms when novel targets and drugs are identified or to discontinue arms with ineffective treatments [5, 8, 10, 14]. However, some researchers argue that the prospect of patient clinical benefit from umbrella trials is limited [15,16,17,18]. Since umbrella trials’ implementation in 2006 [19], many statistical objections [20] and ethical challenges have been identified [8, 21].

A favorable risk-benefit ratio is one of the fundamental ethical requirements of conducting research with human participants [22, 23]. The evaluation of risks and potential benefits to study participants requires a careful ethical analysis based on relevant data [24, 25]. The safety and toxicity rates of anticancer agents in standard phase I–III clinical trials have already been estimated [26,27,28]. The recent analyses were focused on targeted therapies [29, 30] which play an important role in precision medicine [31]; the performance of targeted therapies is enhanced when used in combination with cytotoxic drugs [32]. However, the only RCT in precision oncology was negative for survival [33]. Yet, little is known about the risk-benefit profile for umbrella oncology trials. The objective of our systematic review with meta-analysis was to evaluate the risks and benefits in umbrella clinical trials testing targeted drugs or a combination of targeted agents with chemotherapy. Specifically, our analysis addresses four issues: (1) the utility of a new strategy of clinical trials (umbrella designs) in oncology, (2) the utility of precision oncology, (3) the utility of pooling populations across arms and across chemotherapies, and (4) the likelihood that a drug works in more than one specific population.

Methods

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines [34, 35].

Eligibility criteria

The inclusion and exclusion criteria were defined prospectively in the study protocol [36], and they are summarized in Table 1. The key inclusion criteria were as follows: (1) cancer umbrella clinical trials as defined by the American Society of Clinical Oncology [4], platform umbrella trials, or sub-studies being a part of the cancer umbrella trial; (2) adult or mixed population studies in which at least 50% participants were ≥ 18 years; (3) patients were diagnosed with any malignancy (solid or hematological) at any stage; and (4) assessment of drug-related toxicity and/or response of targeted therapy drug/s (monoclonal antibodies or small molecules or antibody-drug conjugates [37]) or a combination of targeted therapy with chemotherapy regimens in at least one experimental arm or sub-trial. We excluded studies evaluating the following: (1) hormone therapies, immunotherapies (e.g., monoclonal antibodies that were also immunotherapy), or chemotherapy only regimens; (2) a combination of targeted therapy with immunotherapy (response profile in such combination is much different from targeted solo therapies, making comparisons with regimens included in the review impossible); and (3) non-pharmacological modalities (e.g., radiotherapy, surgery, stem cell therapy, or any of these, except for targeted therapy, combined with surgery).

Table 1 Inclusion and exclusion criteria
Full size table

Data sources and search strategy

We systematically searched Embase and PubMed for umbrella trial articles and abstracts published between 1 January 2006 and 7 October 2019, using strategies that included keywords and suggested MeSH and Emtree entry terms, their synonyms, and closely related words (Additional file 1: Table S1). Searches were not limited by language. The starting date of our search period was determined by the year of launching the first umbrella study [19]. Our search strategies were checked using the Canadian Agency for Drugs and Technologies in Health peer-review checklist for search strategies [38].

Study selection process

Two experienced coders (KS, MTW) independently screened the records for the initial study inclusion and performed a full-text screening to determine the final inclusions. Disagreements were resolved by discussion, and when necessary, a third person, an arbiter, was involved (MW).

Data extraction

We created and piloted a data extraction form. Based on the pilot, we refined and prepared the final version (available from the Open Science Framework (OSF), https://osf.io/kuyaz/). Data were extracted from each publication independently by two reviewers (KS, MB). Discrepancies were resolved by discussion, and when necessary, an arbiter was involved (MW). An experienced medical oncologist had a supervisory role (BG). In the case of multiple publications for the same study, the results from the full publication and/or the most recent version were used in the extraction. If the NCT number was provided, additional information was searched and extracted from ClinicalTrials.gov.

Umbrella trials are very heterogeneous; some of them are studies with multiple arms (Fig. 1A), and others have a hierarchical structure with sub-trials having a unique registration number (Fig. 1B). We extracted data only from the arms or sub-trials testing targeted therapy drugs or a combination of targeted therapy with chemotherapy. If the umbrella trial or sub-trial included a placebo, control group, or non-match arm, data from these arms were extracted separately for further comparison of matched versus non-matched therapy.

Fig. 1
figure 1

Structure types of umbrella trials. A An umbrella trial with multiple arms. Accrual to each arm is based on the biomarker test result. B An umbrella trial with multiple sub-studies. Each sub-study has a separate registration number and may include an experimental arm and a control group

Full size image

We considered the therapy as “matched” to the disease when at least one tested agent was administered based on the specific molecular features of the patient’s tumor, e.g., a drug matched to the specific genetic change. If patients were treated in (1) biomarker-negative sub-study/arm, (2) so-called non-match sub-study/arm (defined as the arm or sub-study recruiting patients that did not match any of the prespecified biomarkers), (3) placebo, or (4) control group testing only chemotherapy agents, we considered these therapies as not matching the specific tumor molecular characteristics.

For each arm or sub-study, we extracted data related to study characteristics (e.g., phase, location, study status), patient characteristics (e.g., number of enrolled and eligible participants, type of malignancy), intervention (e.g., therapy type, agent names), and outcomes (e.g., objective responses, drug-related adverse events). For more details, see our extraction form (https://osf.io/kuyaz/).

Data curation

We defined a “sub-study” of the umbrella trial as a separate trial within the umbrella protocol with a unique registration number provided by the study authors. In cases where the separate registration number was not provided, we used the term “arm.” The glossary of key manuscript terms is presented in the online appendix (Additional file 1: Table S2).

Umbrella trials generally measure short-term clinical outcomes to yield information about preliminary drug efficacy [8]. We included various measures of clinical benefit reported in umbrella trials: we classified the objective response rate (ORR) and progression-free survival (PFS) as proxies of therapeutic benefit and overall survival (OS) as the direct measure of clinical benefit. We defined the objective response rate as the proportion of participants with partial and/or complete response (reported separately or as an objective response rate) as defined by the study authors. For PFS and OS analyses, we used medians provided by the study authors.

Risks were assessed in terms of patients experiencing severe adverse events, such as the proportion of participants experiencing grade 3, 4, or 5 drug-related AEs as defined by the Common Toxicity Criteria for Adverse Events, version 5.0 (and earlier versions) [39]. An AE was considered as related to the study drug if it was clearly stated by the study authors; expressions such as “AEs attributed to treatment” and “AEs possibly, probably, or definitely related to study drug” were also acceptable. In cases where an event was not clearly described as treatment-related, we excluded it from our risk analysis.

Risk of bias assessment

Two authors (KS, MB) independently assessed the risk of bias for all included studies using the Cochrane risk of bias tools for randomized or non-randomized studies [40, 41]. Every sub-trial/arm was assessed separately by reading all relevant literature. Judgments were based on the algorithms proposed by the authors of ROBINS and RoB2 tools, adjusted to fit the specific aspects of our analysis. Disagreements were resolved by discussion.

Statistical analysis

Objective response rates, treatment-related fatal (grade 5) AE rates, and treatment-related grade 3/4 AEs rates were calculated as the number of each of these outcomes in the sub-study/arm divided by the total number of patients evaluated for response or toxicity in that sub-study/arm. Standard errors and confidence intervals (CIs) for a single proportion were derived. Pooled rates were estimated using meta-analysis for proportion. Modeling with random effects and the restricted maximum likelihood (REML) estimator were used to account for between-study heterogeneity. I2 statistics were calculated to provide a measure of the proportion of overall variation attributable to between-study heterogeneity. Meta-regression was used to explore potential sources of heterogeneity in rates related to (1) categories of therapy type in experimental sub-trials/arms and (2) types of sub-trials/arms (experimental vs non-match/control/placebo), as well as (3) study definition and (4) number of drugs tested. The results are presented as rates with 95% CI in each category and p value from the Q test for heterogeneity in meta-regression. The average number of treatment-related grade 3 and 4 AEs per person with a 95% confidence interval was estimated using a Poisson regression model. Unweighted median with 95% CI was calculated for PFS and OS by bootstrap methods [42] using the “boot” package in the R software.

Meta-analysis was conducted using the metafor package (R version 3.2.3); p < 0.05 was considered statistically significant. All tests were 2-sided.

Results

We retrieved 6207 references from searching databases. After duplicate removal, we screened 4738 records, from which we reviewed 215 full-text documents. In the next step, we searched references of the initially included studies and ClinicalTrials.gov entries and found 2 extra articles. Finally, we included 29 records of 31 sub-trials or arms of 9 umbrella trials. Figure 2 summarizes the search results and reasons for exclusion. A full list of included studies with therapy type, malignancy names, and agents tested is presented in the online appendix (Additional file 1: Table S3) [43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71].

Fig. 2
figure 2

PRISMA 2020 flow diagram. The number of identified, screened, and included records with the exclusion reasons for potentially eligible reports

Full size image

Trial and patient characteristics

We included 31 sub-trials or arms of nine umbrella trials (N = 1637). Six out of 9 umbrella trials can be also classified as platform umbrella trials (Additional file 1: Table S3). The majority (19; 61.2%) of included studies were sub-studies with a separate registration number, ten (32.3%) were umbrella trial arms without a separate registration number, and two (6.5%) were arms of the included sub-studies (Table 2). The majority (27; 87.1%) were biomarker-based experimental sub-studies/arms, two (6.5%) were non-biomarker specific, and one (3.2%) was either a placebo or a control group. Twenty-one sub-studies/arms (67.8%) tested targeted therapies, eight (25.8%) tested a combination of targeted therapy with chemotherapy, one (3.2%) tested standard-of-care chemotherapy, and one (3.2%) was a placebo arm. The majority of included sub-trials/arms (25; 80.6%) were phase II studies and tested one investigational drug (22; 71.0%). Six sub-trials/arms (19.4%) were funded by private sponsors, four (12.9%) by public institutions, and 21 (67.7%) by both private and public sponsors. Thirteen sub-trials/arms (42.0%) were conducted in North America, 12 (38.7%) in Asia, five (16.1%) in Europe, and one (3.2%) in Australia.

Table 2 Characteristics of the included studies
Full size table

In 13 sub-studies/arms, the median age of participants was below 65 years; in 5 sub-studies/arms, the median age was 65 years or higher; and in the remaining 13 sub-studies/arms, the median age was not reported (Table 2). All sub-trials/arms involved only patients with solid tumors.

Benefit in experimental sub-trials/arms

Twenty-two of 27 experimental sub-trials/arms reported response data. We identified 185 objective responses (including 142 partial and 5 complete reported separately and 38 reported as objective responses) among 879 participants evaluated for response. One targeted therapy experimental arm was excluded from the meta-analysis because only 1 patient was evaluated for response in that arm.

The pooled ORR across 21 sub-trials/arms (878 patients) was 19.7% (95% CI 10.5–28.8; I2 = 97.3%; Fig. 3). The ORR for targeted therapies was significantly lower than the ORR for combination of targeted therapy drugs with chemotherapy: 13.3% (95% CI 4.6–21.9) vs 39.0% (95% CI 21.3–56.8), p = 0.005.

Fig. 3
figure 3

Forest plot of proportions of objective response rates in experimental sub-trials/arms included in the meta-analysis (random effects). The analysis included data from 21 experimental sub-trials/arms with a total of 185 objective responses among 878 participants evaluated for response. CI, confidence interval

Full size image

The median PFS ranging from 1.2 to 17.0 months was reported in 16 experimental sub-trials/arms [72]. The pooled median PFS was 2.5 months (95% CI 2.0–7.8).

The median OS ranging from 4.7 to 10.4 months was reported in 12 experimental sub-trials/arms. The pooled median OS was 6.8 months (95% CI 6.0–8.0). We did not compare pooled PFS and OS between targeted therapy and a combination of targeted therapy with chemotherapy because only one sub-study/arm testing combination of the therapies reported these outcomes.

Risk in experimental sub-trials/arms

We analyzed 9 drug-related grade 5 AEs among 999 participants evaluated for toxicity in 15 experimental sub-trials/arms (including 9 drug-related deaths in 5 experimental sub-trials/arms and in the remaining 10 sub-trials/arms the drug-related deaths were reported to be 0). The pooled drug-related death rate across these sub-trials/arms was 0.7% (95% CI 0.1–1.2; I2 = 4.5%; Fig. 4).

Fig. 4
figure 4

Forest plot of proportions of treatment-related grade 5 AEs in experimental sub-trials/arms included in the meta-analysis (random effects). The analysis included data from 15 experimental sub-trials/arms with a total of 9 drug-related deaths among 999 participants evaluated for toxicity. AEs, adverse events; CI, confidence interval

Full size image

The pooled drug-related grade 5 AE rate for 12 experimental targeted therapy sub-trials/arms (among 502 participants evaluated for toxicity) was 1.1% (95% CI 0.2–2.0) and for 3 experimental targeted therapy combined with chemotherapy sub-trials/arms (497 participants) was 0.5% (95% CI 0.1–1.2). Due to the small number of events, statistical comparisons between the different groups were not performed.

Ninety-one patients (34.0%; 95% CI 15.2–52.9) experienced treatment-related grade 3/4 AEs in 5 sub-trials (Fig. 5). The treatment-related grade 3/4 rate in four of these sub-trials/arms testing targeted therapy drugs was 42.6% (95% CI 31.2–53.9).

Fig. 5
figure 5

Forest plot of proportions of treatment-related grade 3/4 AEs in experimental sub-trials included in the meta-analysis (random effects). The analysis included data from 5 experimental sub-trials/arms with a total of 91 patients that experienced drug-related grade 3/4 AEs. AEs, adverse events; CI, confidence interval

Full size image

Three hundred eleven drug-related grade 3/4 AEs were reported in 11 experimental sub-trials/arms (10 targeted therapy, one targeted therapy with chemotherapy) among 695 toxicity evaluable patients, with an average drug-related grade 3/4 AE rate per person of 0.45 (95% CI 0.40–0.50).

Overall benefit and risk

Twenty-five of 31 sub-trials/arms reported 212 objective responses (including 169 partial and 5 complete reported separately and 38 reported as objective responses) among 1148 participants evaluated for response. One arm was excluded from the meta-analysis because only 1 patient was evaluated for response in that arm. The pooled overall ORR across 24 sub-trials/arms (1147 patients) was 17.7% (95% CI 9.5–25.9; I2 = 97.3%; Fig. 6). We did not find a significant difference in ORR between experimental sub-trials/arms versus non-matched therapies: 19.7% (95% CI 10.5–28.8) vs 7.1% (95% CI 0.0–13.5); p = 0.25.

Fig. 6
figure 6

Forest plot of proportions of objective response rates in sub-trials/arms included in the meta-analysis (random effects). The analysis included data from 24 sub-trials/arms with a total of 212 objective responses among 1147 participants evaluated for response. CI, confidence interval

Full size image

The median PFS ranging from 1.2 to 17.0 months was reported in 19 sub-trials/arms. The pooled median PFS was 2.4 months (95% CI 1.9–2.9). For 3 sub-trials/arms with non-matched therapies the pooled median PFS was 2.0 months (95% CI 1.2–3.5).

The median OS ranging from 4.7 to 10.5 months was reported in 13 sub-trials/arms. The pooled median OS was 7.1 months (95% CI 6.1–8.4). We did not compare the pooled OS rates between sub-trial/arm types because there was only one arm reporting this outcome in the non-matched group.

We identified 12 drug-related grade 5 AEs among 1233 patients evaluable for toxicity in 17 sub-trials/arms. The overall pooled drug-related death rate across these sub-trials/arms was 0.8% (95% CI 0.3–1.4; I2 = 7.32%; Fig. 7). We did not compare pooled drug-related death rates between sub-trial/arm types because there were two sub-trials/arms reporting this outcome in the non-match sub-category.

Fig. 7
figure 7

Forest plot of proportions of treatment-related grade 5 AEs in 17 sub-trials/arms included in the meta-analysis (random effects). The analysis included data from 17 sub-trials/arms with a total of 12 drug-related deaths among 1233 participants evaluated for toxicity. AEs, adverse events; CI, confidence interval

Full size image

Treatment-related grade 3/4 AEs were not reported in non-matched sub-trials/arms. The overall treatment-related grade 3/4 AE rate and the average drug-related grade 3/4 AE rate per person remain the same as for the experimental arms described previously.

Sub-group analysis of benefit and risk

We observed significant differences in ORR between Phase II and Phase III trials: 14.0% (95% CI 6.9-21.2) vs 36.4% (95% CI 3.4-69.3); p = 0.03 (Additional file 1: Table S4). Other benefit and risk measures in sub-categories are shown in the online appendix (Additional file 1: Table S4).

Risk of bias assessment

The risk of bias of the included sub-studies/arms is available in the online appendix (Figs. S1-S4). There were two randomized sub-studies with “low” or “some concerns” risk of bias among all domains. There were 27 nonrandomized sub-trials/arms, and 16 of them (59%) were assessed as having the overall risk of bias as “serious” or “critical.” High levels of bias were mainly due to bias in the selection of the reported result.

Discussion

To our knowledge, we report the first systematic review and meta-analysis providing risk and benefit estimates for cancer umbrella trials testing targeted therapies or a combination of targeted therapies with chemotherapy. These analyses include four aspects: (1) the utility of a new strategy of clinical trials (umbrella designs) in oncology, (2) the utility of precision oncology, (3) the utility of pooling populations across arms and across chemotherapies, and (4) the likelihood that a drug works in more than one specific population.

Maximizing benefit and minimizing harm for participants of clinical research are one of the crucial ethical requirements. Umbrella trial designs are believed not only to be more flexible than traditional designs by allowing simultaneous evaluation of multiple treatment options but also to have a better risk/benefit ratio for trial participants [10]. However, our findings do not support the expectations of an increased benefit/risk ratio for participants of cancer umbrella trials.

Normally, the risk-benefit ratio in clinical trials is a function of the drug. However, clinical trial design may also affect this equation. Some elements of the umbrella design are expected to provide a more favorable risk/benefit ratio for participants than the classical trial designs. For example, the “precision medicine” approach of assigning patients to treatment arms based on the genetic characteristics of their tumor may lead to an expectation of higher responses and lower adverse event rates. Moreover, most of the umbrella trials in our study used the platform design and an adaptive design. This means that more patients may be enrolled in an arm with more promising health benefits, during the course of the study. Our results cast doubts on such an assumption. For example, the majority of the sub-trials/arms in our sample were phase II trials with pooled ORR of 14.0%. This result is similar to the pooled ORR in the previous meta-analysis of phase II single-agent studies (12.7%) [27] but lower than the overall ORR in eight cancer basket trials published until 31 March 2018 (25%) [73].

Our findings suggest that in all umbrella clinical trials in oncology published before 7 October 2019, the chances of responding from targeted therapies were lower than from the combination of targeted therapies with chemotherapy. This observation is consistent with the previous findings showing the increased overall response rates in trials testing a combination of targeted therapies with cytotoxic drugs [32]. Unfortunately, a comparison of the overall benefits and risks in our meta-analysis to other studies is limited as the included sub-trials/arms were very heterogeneous, e.g., they were of different phases, and included heterogeneous populations and various types of cancers.

The majority of the included sub-trials/arms reported surrogate outcomes: ORR (25; 80.6%) or PFS (19; 61.3%). Thirteen sub-trials/arms (41.9%) reported OS. A variety of endpoints makes a comparison of outcomes in meta-research more demanding. Importantly, although sometimes accepted by regulatory agencies for approval, ORR and PFS are surrogate markers that have shown poor correlation with OS and quality of life in most tumor types. Furthermore, early phases of clinical trials also poorly predict phase III success. Therefore, surrogate measures should be considered only hypothesis-generating and not a marker of true clinical benefit [74,75,76,77,78].

We did not find significant differences in objective response rates between therapies matched to the specific cancer biomarkers versus non-matched therapies or controls. This finding may suggest that the approaches to maximize the direct benefit in umbrella trials, e.g., genome-driven stratification and assignment to the most promising arm, may not be sufficient to deliver the appropriate therapy matching the heterogeneous and mutable tumor [8]. This also raises the question if biomarkers used to define the target in precision oncology may be suboptimal [79]. However, other systematic reviews analyzing pediatric phase I [80] and phase II [27] cancer trials showed high objective response rates in trials with target-specific enrolment [80] or in trials adopting personalized treatment approach [27].

The drug-related death rate of 1.1% in phase II sub-trials/arms in our sample (Additional file 1: Table S4) is also similar to the drug-related grade 5 AEs rate in the previous meta-analysis of phase II single-agent studies that used personalized strategy (1.5%) [27]. This may indicate that phase II umbrella trials do not offer a lower risk for trial participants than classical phase II clinical trials.

Debates about whether precision medicine is an illusion or an objective reality continues in oncology [81]. Our findings do not support the expectation of increased participants’ benefit in cancer umbrella trials. Patients should be clearly informed that the majority of participants (82.3%) of the first launched umbrella trials testing targeted therapy agents or combination of targeted therapy drugs with chemotherapy did not respond to a given therapy.

The objective of our study was to analyze the risk/benefit ratio for umbrella clinical trials testing targeted drugs or a combination of targeted therapy with chemotherapy. Analyses of that type are crucial sources of information for participants, researchers, ethics committee members, and other stakeholders and decision-makers. When performing our analysis, we found that the complexity of the umbrella design and the low quality of reporting makes a comparison of the results from trials and sub-trials very difficult. Many umbrella arms and sub-trials of the umbrella trials included in our study were closed without any explanation and without a report of the results. Our findings may be used to improve umbrella trials design reporting.

Limitations

Our study should be interpreted in light of the following limitations.

First, we included all umbrella trials or sub-trials being a part of one umbrella trial in which one cancer type was divided into sub-types to test the different drugs. Because umbrella trials are very heterogeneous and have a hierarchical structure and every arm may be of different phase, we did not compare the whole umbrella trials but compared the sub-trials and arms. We created this novel methodology to analyze the outcomes of a complex umbrella trial design but our methods may be further modified and improved.

Second, we observed inconsistency in reporting the outcomes and selective reporting of the results in the majority of the included umbrella trials. For example, in the VIKTORY trial [71], outcomes were not provided for six out of 10 arms. Low quality of reporting of umbrella trials is a serious issue not only because of the difficulties in performing meta-research but also for other ethical reasons, including patients’ safety and decision-making process in designing new trials.

Third, ORR and PFS reported in most umbrella trials are considered a surrogate benefit and are not markers of direct clinical benefit [76, 82]. We analyze them because they are the best available surrogates of clinical benefit.

Fourth, our systematic review is restricted to umbrella trials which results were reported between 2006 and 2019 and does not include trials developed based on the findings of those initial trials. As relatively novel designs, umbrella trials are expected to improve methodologically over time and may produce results leading to different conclusions than those presented here.

Fifth, the limited number of eligible sub-trials reporting the outcomes of interest did not allow for all possible comparisons, for example, pooled PFS and OS.

Sixth, the majority of the analyzed studies reported summary results. Thus, we could not test whether risk and benefit in umbrella trials depend on the line of treatment or cancer histology.

Conclusions

This is the first systematic review with meta-analysis assessing the risk and benefits of umbrella clinical trials. We found that the overall objective response rate in umbrella trials testing targeted drugs or a combination of targeted therapy with chemotherapy was 17.7%, and the overall drug-related death rate was 0.8%. Patients enrolling in umbrella trials should be clearly informed about the risk and benefit predictions for these trials. Our findings do not support the expectation of increased patients’ benefits in cancer umbrella trials. Further studies should investigate whether umbrella trial design and precision oncology approach improve patient outcomes. Our study identified serious problems with reporting and transparency of umbrella design which may undermine a promise of more efficient and patient-centered trials.

Registration and protocol

The study protocol was prospectively registered in PROSPERO (CRD42020171494) [36].

Availability of data and materials

Data are available in the Open Science Framework online public database (https://osf.io/uc5ds/). Additional data are available upon request.

Abbreviations

AEs:

Adverse events

CI:

Confidence interval

ECOG:

Eastern Cooperative Oncology Group

OS:

Overall survival

ORR:

Objective response rate

OSF:

Open Science Framework

PFS:

Progression-free survival

REML:

Restricted maximum likelihood

WHO:

World Health Organization

References

  1. Senft D, Leiserson MDM, Ruppin E, Ronai ZA. Precision oncology: the road ahead. Trends Mol Med. 2017;23(10):874–98. https://doi.org/10.1016/j.molmed.2017.08.003.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Siden EG, Park JJH, Zoratti MJ, Dron L, Harari O, Thorlund K, et al. Reporting of master protocols towards a standardized approach: a systematic review. Contemp Clin Trials Commun. 2019;15:100406. https://doi.org/10.1016/j.conctc.2019.100406.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Park JJH, Hsu G, Siden EG, Thorlund K, Mills EJ. An overview of precision oncology basket and umbrella trials for clinicians. CA Cancer J Clin. 2020;70(2):125–37. https://doi.org/10.3322/caac.21600.

    Article  PubMed  PubMed Central  Google Scholar 

  4. American Society of Clinical Oncology Research Community Forum. Library of resources: conducting and managing cancer clinical trials. Alexandria: American Society of Clinical Oncology; 2020. https://www.asco.org/sites/new-www.asco.org/files/content-files/research-and-progress/documents/2020-ASCO-RCF-Library-Methodology.pdf. Accessed 3 Dec 2021

    Google Scholar 

  5. U. S Department of Health and Human Services Food and Drug Administration, master protocols: efficient clinical trial design strategies to expedite development of oncology drugs and biologics guidance for industry (Draft Guidance) 2018. https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM621817.pdf. Accessed 3 Dec 2021.

  6. West HJ. Novel precision medicine trial designs. JAMA Oncol. 2017;3(3):423. https://doi.org/10.1001/jamaoncol.2016.5299.

    Article  PubMed  Google Scholar 

  7. Renfro LA, Mandrekar SJ. Definitions and statistical properties of master protocols for personalized medicine in oncology. J Biopharm Stat. 2018;28(2):217–28. https://doi.org/10.1080/10543406.2017.1372778.

    Article  PubMed  Google Scholar 

  8. Strzebonska K, Waligora M. Umbrella and basket trials in oncology: ethical challenges. BMC Med Ethics. 2019;20(1):58. https://doi.org/10.1186/s12910-019-0395-5.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Woodcock J, LaVange LM. Master protocols to study multiple therapies, multiple diseases, or both. N Engl J Med. 2017;377(1):62–70. https://doi.org/10.1056/NEJMra1510062.

    Article  CAS  PubMed  Google Scholar 

  10. Waligora M, Strzebonska K. Umbrella trials design. In: Frey B, editor. The SAGE encyclopedia of research design. USA: SAGE Publications, Inc; 2022. p. 1763–4. https://doi.org/10.4135/9781071812082.n657.

  11. Bogin V. Master protocols: new directions in drug discovery. Contemp Clin Trials Commun. 2020;18:100568. https://doi.org/10.1016/j.conctc.2020.100568.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Verweij J, Hendriks HR, Zwierzina H. Innovation in oncology clinical trial design. Cancer Treat Rev. 2019;74:15–20. https://doi.org/10.1016/j.ctrv.2019.01.001.

    Article  CAS  PubMed  Google Scholar 

  13. Renfro LA, Sargent DJ. Statistical controversies in clinical research: basket trials, umbrella trials, and other master protocols: a review and examples. Ann Oncol. 2017;28(1):34–43. https://doi.org/10.1093/annonc/mdw413.

    Article  CAS  PubMed  Google Scholar 

  14. National Cancer Institute. NCI Dictionaries. Dictionary of cancer terms. Umbrella trial. https://www.cancer.gov/publications/dictionaries/cancer-terms/def/umbrella-trial. Accessed 3 Dec 2021.

  15. Prasad V. Perspective: the precision-oncology illusion. Nature. 2016;537(7619):S63. https://doi.org/10.1038/537S63a.

    Article  CAS  PubMed  Google Scholar 

  16. Tannock IF, Hickman JA. Limits to personalized cancer medicine. N Engl J Med. 2016;375(13):1289–94. https://doi.org/10.1056/NEJMsb1607705.

    Article  PubMed  Google Scholar 

  17. Voest EE, Bernards R. DNA-guided precision medicine for cancer: a case of irrational exuberance? Cancer Discov. 2016;6(2):130–2. https://doi.org/10.1158/2159-8290.CD-15-1321.

    Article  CAS  PubMed  Google Scholar 

  18. Kimmelman J, Tannock I. The paradox of precision medicine. Nat Rev Clin Oncol. 2018;15(6):341–2. https://doi.org/10.1038/s41571-018-0016-0.

    Article  PubMed  Google Scholar 

  19. Park JJH, Siden E, Zoratti MJ, Dron L, Harari O, Singer J, et al. Systematic review of basket trials, umbrella trials, and platform trials: a landscape analysis of master protocols. Trials. 2019;20(1):572. https://doi.org/10.1186/s13063-019-3664-1.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Collignon O, Gartner C, Haidich AB, Hemmings RJ, Hofner B, Pétavy F, et al. Current statistical considerations and regulatory perspectives on the planning of confirmatory basket, umbrella, and platform trials. Clin Pharmacol Ther. 2020;107(5):1059–67. https://doi.org/10.1002/cpt.1804.

    Article  PubMed  Google Scholar 

  21. Hey SP, Weijer C, Taljaard M, Kesselheim AS. Research ethics for emerging trial designs: does equipoise need to adapt? BMJ. 2018;360:k226. https://doi.org/10.1136/bmj.k226.

    Article  PubMed  Google Scholar 

  22. International Ethical Guidelines for Health-related Research Involving Humans, Fourth Edition. Geneva: Council for International Organizations of Medical Sciences (CIOMS); 2016. https://cioms.ch/wp-content/uploads/2017/01/WEB-CIOMS-EthicalGuidelines.pdf. Accessed 7 Jun 2022.

  23. Rid A, Wendler D. A framework for risk-benefit evaluations in biomedical research. Kennedy Inst Ethics J. 2011;21(2):141–79. https://doi.org/10.1353/ken.2011.0007.

    Article  PubMed  Google Scholar 

  24. Weijer C, Miller PB. When are research risks reasonable in relation to anticipated benefits? Nat Med. 2004;10(6):570–3. https://doi.org/10.1038/nm0604-570.

    Article  CAS  PubMed  Google Scholar 

  25. Wendler D, Miller FG. Assessing research risks systematically: the net risks test. J Med Ethics. 2007;33(8):481–6. https://doi.org/10.1136/jme.2005.014043.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Waligora M, Bala MM, Koperny M, Wasylewski MT, Strzebonska K, Jaeschke R, et al. Risk and surrogate benefit for pediatric phase I trials in oncology: a systematic review with meta-analysis. PLoS Med. 2018;15(2):e1002505. https://doi.org/10.1371/journal.pmed.1002505.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Schwaederle M, Zhao M, Lee JJ, Eggermont AM, Schilsky RL, Mendelsohn J, et al. Impact of precision medicine in diverse cancers: a meta-analysis of phase II clinical trials. J Clin Oncol. 2015;33(32):3817–25. https://doi.org/10.1200/JCO.2015.61.5997.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Magee DE, Hird AE, Klaassen Z, Sridhar SS, Nam RK, Wallis CJD, et al. Adverse event profile for immunotherapy agents compared with chemotherapy in solid organ tumors: a systematic review and meta-analysis of randomized clinical trials. Ann Oncol. 2020;31(1):50–60. https://doi.org/10.1016/j.annonc.2019.10.008.

    Article  CAS  PubMed  Google Scholar 

  29. Mackley MP, Fernandez NR, Fletcher B, Woolcott CG, Fernandez CV. Revisiting risk and benefit in early oncology trials in the era of precision medicine: a systematic review and meta-analysis of phase I trials of targeted single-agent anticancer therapies. JCO Precis Oncol. 2021;5:17–26. https://doi.org/10.1200/PO.20.00214.

    Article  PubMed  Google Scholar 

  30. Strzebonska K, Wasylewski MT, Zaborowska L, Polak M, Slugocka E, Stras J, et al. Risk and benefit for targeted therapy agents in pediatric phase II trials in oncology: a systematic review with a meta-analysis. Target Oncol. 2021;16(4):415–24. https://doi.org/10.1007/s11523-021-00822-5.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Bedard PL, Hyman DM, Davis MS, Siu LL. Small molecules, big impact: 20 years of targeted therapy in oncology. Lancet. 2020;395(10229):1078–88. https://doi.org/10.1016/S0140-6736(20)30164-1.

    Article  CAS  PubMed  Google Scholar 

  32. Azam F, Vazquez A. Trends in phase II trials for cancer therapies. Cancers (Basel). 2021;13(2):178. https://doi.org/10.3390/cancers13020178.

    Article  CAS  Google Scholar 

  33. Le Tourneau C, Belin L, Paoletti X, Biéche I, Kamal M. Precision medicine: lessons learned from the SHIVA trial – author’s reply. Lancet Oncol. 2015;16(16):e581–2. https://doi.org/10.1016/S1470-2045(15)00455-6.

    Article  PubMed  Google Scholar 

  34. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. https://doi.org/10.1136/bmj.n71.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;372:n160. https://doi.org/10.1136/bmj.n160.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Strzebonska K, Wasylewski M, Blukacz M, Gyawali B, Waligora M. Risk and benefit for umbrella trials in oncology: a systematic review with meta-analysis: PROSPERO; 2020. https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020171494. Accessed 3 Dec 2021

    Google Scholar 

  37. Karp DD, Falchook GS. Handbook of targeted cancer therapy and immunotherapy. Targeted cancer therapy. 2nd ed. Philadelphia: Wolters Kluwer; 2018.

    Google Scholar 

  38. Canadian Agency for Drugs and Technologies in Health. CADTH peer review checklist for search strategies. Ottawa: The Agency; 2013. https://www.cadth.ca/sites/default/files/is/Peer_review/CADTH%20Peer%20Review%20Checklist%20for%20Search%20Strategies_e.pdf. Accessed 3 Dec 2021

    Google Scholar 

  39. National Cancer Institute. Common Terminology Criteria for Adverse Events: (CTCAE). https://ctep.cancer.gov/protocoldevelopment/electronic_applications/ctc.htm. Accessed 3 Dec 2021.

  40. Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomized trials. BMJ. 2019;366:l4898. https://doi.org/10.1136/bmj.l4898.

    Article  PubMed  Google Scholar 

  41. Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomized studies of interventions. BMJ. 2016;355:i4919. https://doi.org/10.1136/bmj.i4919.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Carpenter J, Bithell J. Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Stat Med. 2000;19(9):1141–64 https://doi.org/10.1002/(sici)1097-0258(20000515)19:9<1141::aid-sim479>3.0.co;2-f.

    Article  CAS  Google Scholar 

  43. Kim ES, Herbst RS, Wistuba II, Lee JJ, Blumenschein GR Jr, Tsao A, et al. The BATTLE trial: personalizing therapy for lung cancer. Cancer Discov. 2011;1(1):44–53. https://doi.org/10.1158/2159-8274.CD-10-0010.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Tsao AS, Liu S, Lee JJ, Alden C, Blumenschein G, Herbst R, et al. Clinical outcomes and biomarker profiles of elderly pretreated NSCLC patients from the BATTLE trial. J Thorac Oncol. 2012;7(11):1645–52. https://doi.org/10.1097/JTO.0b013e31826910ff.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Blumenschein GR Jr, Saintigny P, Liu S, Kim ES, Tsao AS, Herbst RS, et al. Comprehensive biomarker analysis and final efficacy results of sorafenib in the BATTLE trial. Clin Cancer Res. 2013;19(24):6967–75. https://doi.org/10.1158/1078-0432.CCR-12-1818.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Herbst RS, Blumenschein GR Jr, Kim ES, Lee J, Tsao AS, Alden CM, et al. Sorafenib treatment efficacy and KRAS biomarker status in the Biomarker-Integrated Approaches of Targeted Therapy for Lung Cancer Elimination (BATTLE) trial. J Clin Oncol. 2010;28(15_suppl):7609. https://doi.org/10.1200/jco.2010.28.15_suppl.7609.

    Article  Google Scholar 

  47. Tsao AS, Liu S, Lee JJ, Alden CM, Blumenschein GR Jr, Herbst R, et al. Clinical and biomarker outcomes of the phase II vandetanib study from the BATTLE trial. J Thorac Oncol. 2013;8(5):658–61. https://doi.org/10.1097/JTO.0b013e31828d08ae.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Papadimitrakopoulou V, Lee JJ, Wistuba II, Tsao AS, Fossella FV, Kalhor N, et al. The BATTLE-2 Study: a biomarker-integrated targeted therapy study in previously treated patients with advanced non-small-cell lung cancer. J Clin Oncol. 2016;34(30):3638–47. https://doi.org/10.1200/JCO.2015.66.0084.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Papadimitrakopoulou V, Lee JJ, Wistuba II, Tsao AS, Fossella FV, Heymach J, et al. BATTLE-2: KRAS mutation and outcome in a biomarker-integrated study in previously treated patients (pts) with advanced non-small cell lung cancer (NSCLC). J Clin Oncol. 2014;32(15_suppl):8042. https://doi.org/10.1200/jco.2014.32.15_suppl.8042.

    Article  Google Scholar 

  50. Zhou Q, Zhang XC, Tu HY, Gan B, Wang BC, Xu CR, et al. Biomarker-integrated study of single agent targeting molecular alterations of PI3KCA, MET, ALK, ROS1, KRAS, NRAS or BRAF in advanced NSCLC: phase 2 umbrella trial in China (CTONG1505). Ann Oncol. 2018;29(9_suppl):ix113. https://doi.org/10.1093/annonc/mdy441.

    Article  Google Scholar 

  51. Wu Y, Zhou Q, Zhang X, Tu H, Gan B, Wang B, et al. P1. 01-97 cluster trial: Ph2 biomarker-integrated study of single agent alpelisib, capmatinib, ceritinib and binimetinib in advNSCLC. J Thorac Oncol. 2018;13(10):S500–1. https://doi.org/10.1016/j.jtho.2018.08.653.

    Article  Google Scholar 

  52. Adams RA, Brown E, Brown L, Butler R, Falk S, Fisher D, et al. FOCUS4-D: results from a randomised, placebo controlled trial (RCT) of AZD8931 (an inhibitor of signalling by HER1, 2, and 3) in patients (pts) with advanced or metastatic colorectal cancer (aCRC) in tumours that are wildtype (wt) for BRAF, PIK3CA, KRAS & NRAS. Ann Oncol. 2016;27(6_suppl):vi168. https://doi.org/10.1093/annonc/mdw370.57.

    Article  Google Scholar 

  53. Richman SD, Adams R, Brown E, Brown L, Butler R, Falk S, et al. FOCUS4: MAMS trial design in action. Early closure of FOCUS4-D (Pan-HER 1, 2 and 3 inhibitor versus placebo) in advanced colorectal cancer (aCRC) patients, with tumours wildtype (WT) for KRAS, NRAS, BRAF and PIK3CA. J Pathol. 2017;243(S1):S13.

    Google Scholar 

  54. Adams R, Brown E, Brown L, Butler R, Falk S, Fisher D, et al. Inhibition of EGFR, HER2, and HER3 signalling in patients with colorectal cancer wild-type for BRAF, PIK3CA, KRAS, and NRAS (FOCUS4-D): a phase 2–3 randomised trial. Lancet Gastroenterol Hepatol. 2018;3(3):162–71. https://doi.org/10.1016/S2468-1253(17)30394-1.

    Article  PubMed  Google Scholar 

  55. Herbst R, Redman M, Gandara DR, Hirsch FR, Mack P, Borghaei H, et al. OA 14.07 progress in lung squamous cell carcinoma from the Lung-MAP Master Protocol (S1400) sub-studies S1400A, S1400B, S1400C and S1400D. J Thorac Oncol. 2017;12(11):S1783–4. https://doi.org/10.1016/j.jtho.2017.09.412.

    Article  Google Scholar 

  56. Wade JL, Langer CJ, Redman M, Aggarwal C, Bradley JD, Crawford J, et al. A phase II study of GDC-0032 (taselisib) for previously treated PI3K positive patients with stage IV squamous cell lung cancer (SqNSCLC): LUNG-MAP sub-study SWOG S1400B. J Clin Oncol. 2017;35(15_suppl):9054. https://doi.org/10.1200/JCO.2017.35.15_suppl.9054.

    Article  Google Scholar 

  57. Langer CJ, Redman MW, Wade JL 3rd, Aggarwal C, Bradley JD, Crawford J, et al. SWOG S1400B (NCT02785913), a phase II study of GDC-0032 (taselisib) for previously treated PI3K-positive patients with stage IV squamous cell lung cancer (Lung-MAP Sub-Study). J Thorac Oncol. 2019;14(10):1839–46. https://doi.org/10.1016/j.jtho.2019.05.029.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  58. Edelman MJ, Redman MW, Albain KS, McGary EC, Rafique N, Petro DP, et al. A phase II study of palbociclib (P) for previously treated cell cycle gene alteration positive patients (pts) with stage IV squamous cell lung cancer (SCC): Lung-MAP sub-study SWOG S1400C. J Clin Oncol. 2017;35(15_suppl):9056. https://doi.org/10.1200/JCO.2017.35.15_suppl.9056.

    Article  Google Scholar 

  59. Edelman MJ, Redman MW, Albain KS, McGary EC, Rafique NM, Petro D, et al. SWOG S1400C (NCT02154490) - a phase II study of palbociclib for previously treated cell cycle gene alteration–positive patients with stage IV squamous cell lung cancer (Lung-MAP Substudy). J Thorac Oncol. 2019;14(10):1853–9. https://doi.org/10.1016/j.jtho.2019.06.027.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Aggarwal C, Redman MW, Lara P, Borghaei H, Hoffman PC, Bradley JD, et al. Phase II study of the FGFR inhibitor AZD4547 in previously treated patients with FGF pathway-activated stage IV squamous cell lung cancer (SqNSCLC): LUNG-MAP sub-study SWOG S1400D. J Clin Oncol. 2017;35(15_suppl):9055. https://doi.org/10.1200/JCO.2017.35.15_suppl.9055.

    Article  Google Scholar 

  61. Aggarwal C, Redman MW, Lara PN Jr, Borghaei H, Hoffman P, Bradley JD, et al. SWOG S1400D (NCT02965378), a phase II study of the fibroblast growth factor receptor inhibitor AZD4547 in previously treated patients with fibroblast growth factor pathway–activated stage IV squamous cell lung cancer (Lung-MAP Substudy). J Thorac Oncol. 2019;14(10):1847–52. https://doi.org/10.1016/j.jtho.2019.05.041.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  62. Owonikoko TK, Redman MW, Byers LA, Hirsch FR, Mack PC, Schwartz LH, et al. A phase II study of talazoparib (BMN 673) in patients with homologous recombination repair deficiency (HRRD) positive stage IV squamous cell lung cancer (Lung-MAP Sub-Study, S1400G). J Clin Oncol. 2019;37:15_suppl–9022. https://doi.org/10.1200/JCO.2019.37.15_suppl.9022.

    Article  Google Scholar 

  63. Waqar SN, Redman MW, Arnold SM, Hirsch FR, Mack PC, Schwartz LH, et al. Phase II study of ABBV-399 (process II) in patients with C-MET positive stage IV/recurrent lung squamous cell cancer (SCC): LUNG-MAP sub-study S1400K (NCT03574753). J Clin Oncol. 2019;37:15_suppl–9075. https://doi.org/10.1200/JCO.2019.37.15_suppl.9075.

    Article  Google Scholar 

  64. Seymour MT, Brown SR, Middleton G, Maughan T, Richman S, Gwyther S, et al. Panitumumab and irinotecan versus irinotecan alone for patients with KRAS wild-type, fluorouracil-resistant advanced colorectal cancer (PICCOLO): a prospectively stratified randomised trial. Lancet Oncol. 2013;14(8):749–59. https://doi.org/10.1016/S1470-2045(13)70163-3.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  65. Park S, Shim J, Jung HA, Sun JM, Lee SH, Park WY, et al. Biomarker driven phase II umbrella trial study of AZD1775, AZD2014, AZD2811 monotherapy in relapsed small cell lung cancer. J Clin Oncol. 2019;37(15_suppl):8514. https://doi.org/10.1200/JCO.2019.37.15_suppl.8514.

    Article  Google Scholar 

  66. Pivot X, Poole C, Martín M, Gligorov J, Barrios CH, Vrdoljak E, et al. The open-label, multinational, multicenter, phase IIIB umbrella study of subcutaneous trastuzumab with or without chemotherapy or pertuzumab in patients (pts) with HER2-positive early breast cancer (EBC) or metastatic breast cancer (MBC): pooled analysis of safety data from the UmbHER1 program. Eur J Cancer. 2018;92(Suppl. 3):S105–6.

    Article  Google Scholar 

  67. Pivot X, Poole C, Martín M, Gligorov J, Barrios CH, Vrdoljak E, et al. An open-label, multinational, multicentre, phase IIIB umbrella study of subcutaneous trastuzumab with or without chemotherapy or pertuzumab in patients with HER2-positive early or metastatic breast cancer (UmbHER1): interim safety results from early breast cancer studies. Ann Oncol. 2016;27:6_suppl–vi63. https://doi.org/10.1093/annonc/mdw364.66.

    Article  Google Scholar 

  68. Zambetti M, Montemurro F, Morandi P, Zamagni C, Brandes AA, Bisagni G, et al. Safety profile of subcutaneous trastuzumab for the treatment of patients with HER2-positive early or locally advanced breast cancer: primary analysis of the SCHEARLY study. Eur J Cancer. 2018;105:61–70. https://doi.org/10.1016/j.ejca.2018.09.034.

    Article  CAS  PubMed  Google Scholar 

  69. Woodward N, De Boer RH, Redfern A, White M, Young J, Truman M, et al. Results from the first multicenter, open-label, phase IIIb study investigating the combination of pertuzumab with subcutaneous trastuzumab and a taxane in patients with HER2-positive metastatic breast cancer (SAPPHIRE). Clin Breast Cancer. 2019;19(3):216–24. https://doi.org/10.1016/j.clbc.2019.02.008.

    Article  CAS  PubMed  Google Scholar 

  70. Lee J, Kim ST, Kozarewa I, Kim HK, Kim K, Mortimer PG, et al. Selumetinib plus docetaxel as second-line chemotherapy in KRAS mutant, KRAS amplified or MEK signatured gastric cancer patients: first arm of the umbrella trial in GC though the molecular screening, VIKTORY trial. J Clin Oncol. 2018;36:15_suppl–4061. https://doi.org/10.1200/JCO.2018.36.15_suppl.4061.

    Article  Google Scholar 

  71. Lee J, Kim ST, Kim K, Lee H, Kozarewa I, Mortimer PGS, et al. Tumor genomic profiling guides patients with metastatic gastric cancer to targeted treatment: the VIKTORY Umbrella Trial. Cancer Discov. 2019;9(10):1388–405. https://doi.org/10.1158/2159-8290.CD-19-0442.

    Article  CAS  PubMed  Google Scholar 

  72. Strzebonska K, Blukacz M, Wasylewski MT, Polak M, Gyawali B, Waligora M. Raw data file for risk and benefit for umbrella trials in oncology: a systematic review and meta-analysis. 2022. The Open Science Framework, https://osf.io/cx2vh/.

    Google Scholar 

  73. Hazim A, Prasad V. A pooled analysis of published, basket trials in cancer medicine. Eur J Cancer. 2018;101:244–50. https://doi.org/10.1016/j.ejca.2018.06.035.

    Article  PubMed  Google Scholar 

  74. Wasylewski MT, Strzebonska K, Koperny M, Polak M, Kimmelman J, Waligora M. Clinical development success rates and social value of pediatric phase 1 trials in oncology. PLoS One. 2020;15(6):e0234911. https://doi.org/10.1371/journal.pone.0234911.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  75. Booth CM, Eisenhauer EA. Progression-free survival: meaningful or simply measurable? J Clin Oncol. 2012;30(10):1030–3. https://doi.org/10.1200/JCO.2011.38.7571.

    Article  PubMed  Google Scholar 

  76. Kovic B, Jin X, Kennedy SA, Hylands M, Pędziwiatr M, Kuriyama A, et al. Evaluating progression-free survival as a surrogate outcome for health-related quality of life in oncology: a systematic review and quantitative analysis. JAMA Intern Med. 2018;178(12):1586–96. https://doi.org/10.1001/jamainternmed.2018.4710.

    Article  PubMed  PubMed Central  Google Scholar 

  77. Gyawali B, Hey SP, Kesselheim AS. Evaluating the evidence behind the surrogate measures included in the FDA’s table of surrogate endpoints as supporting approval of cancer drugs. EClinicalMedicine. 2020;21:100332. https://doi.org/10.1016/j.eclinm.2020.100332.

    Article  PubMed  PubMed Central  Google Scholar 

  78. Hwang TJ, Gyawali B. Association between progression-free survival and patients’ quality of life in cancer clinical trials. Int J Cancer. 2019;144(7):1746–51. https://doi.org/10.1002/ijc.31957.

    Article  CAS  PubMed  Google Scholar 

  79. Gyawali B. Point: the imprecise pursuit of precision medicine: are biomarkers to blame? J Natl Compr Canc Netw. 2017;15(7):859–62. https://doi.org/10.6004/jnccn/2017.0126.

    Article  PubMed  Google Scholar 

  80. Cohen JW, Akshintala S, Kane E, Gnanapragasam H, Widemann BC, Steinberg SM, et al. A systematic review of pediatric phase I trials in oncology: toxicity and outcomes in the era of targeted therapies. Oncologist. 2020;25(6):532–40. https://doi.org/10.1634/theoncologist.2019-0615.

    Article  PubMed  PubMed Central  Google Scholar 

  81. Kaiser J. Is genome-guided cancer treatment hyped? Science. 2018;360(6387):365. https://doi.org/10.1126/science.360.6387.365.

    Article  CAS  PubMed  Google Scholar 

  82. Samuel JN, Booth CM, Eisenhauer E, Brundage M, Berry SR, Gyawali B. Association of quality-of-life outcomes in cancer drug trials with survival outcomes and drug class. JAMA Oncol. 2022:e220864. https://doi.org/10.1001/jamaoncol.2022.0864.

Download references

Acknowledgements

The authors thank Phyllis Zych Budka for the linguistic edits.

Funding

The study was funded by the National Science Center, Poland, UMO-2019/35/N/HS1/00178 (www.ncn.gov.pl). The authors who received the funding: Karolina Strzebonska (principal investigator) and Mateusz Blukacz. The funder had no role in the study design, data collection and analysis, decision to publish, or manuscript preparation. Karolina Strzebonska is supported by the Foundation for Polish Science (FNP). Publication of this article in Open Access was funded by the Priority Research Area qLife under the program “Excellence Initiative – Research University” at the Jagiellonian University in Krakow (06/IDUB/2019/94).

Author information

Authors and Affiliations

  1. Research Ethics in Medicine Study Group (REMEDY), Faculty of Health Sciences, Jagiellonian University Medical College, Kraków, Poland

    Karolina Strzebonska, Mateusz T. Wasylewski, Maciej Polak & Marcin Waligora

  2. Institute of Psychology, University of Silesia, Katowice, Poland

    Mateusz Blukacz

  3. Institute of Public Health, Faculty of Health Sciences, Jagiellonian University Medical College, Kraków, Poland

    Mateusz Blukacz

  4. Department of Epidemiology and Population Studies, Institute of Public Health, Faculty of Health Sciences, Jagiellonian University Medical College, Kraków, Poland

    Maciej Polak

  5. Department of Oncology and the Department of Public Health Sciences, Queen’s University, Kingston, Ontario, Canada

    Bishal Gyawali

Authors
  1. Karolina Strzebonska
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mateusz Blukacz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mateusz T. Wasylewski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maciej Polak
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bishal Gyawali
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marcin Waligora
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MW and KS developed the concept for the research study. BG provided expertise in oncology. KS and MTW performed the abstract screening. KS and MB conducted the full-text screening and data extraction and assessed the risk of bias. MP performed the statistical analyses. MW supervised the project. KS drafted the manuscript and was responsible for the coordination of all aspects of the work. All the authors critically revised the manuscript for important intellectual content and approved the final version.

Corresponding author

Correspondence to Marcin Waligora.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

Marcin Waligora reports personal fees from the Advisory Bioethics Council (Sanofi) outside the submitted work. Karolina Strzebonska, Mateusz Blukacz, Mateusz T. Wasylewski, Maciej Polak, and Bishal Gyawali declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table S1

. Search strategy. Table S2. Glossary of key manuscript terms. Table S3. List and characteristics of included studies. Table S4. Objective response rates and drug-related fatal toxicity rates assessed in subgroups. Fig. S1. Summary risk of bias graph of randomized sub-studies. Fig. S2. Review authors’ judgements about risk of bias of randomized sub-studies. Fig. S3. Summary risk of bias graph of all non-randomized sub-studies/arms. Fig. S4. Review authors’ judgements about risk of bias for non-randomized sub-studies/arms.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Strzebonska, K., Blukacz, M., Wasylewski, M.T. et al. Risk and benefit for umbrella trials in oncology: a systematic review and meta-analysis. BMC Med 20, 219 (2022). https://doi.org/10.1186/s12916-022-02420-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12916-022-02420-2

Keywords

BMC Medicine

ISSN: 1741-7015

Contact us