Effectiveness of COVID-19 vaccines against SARS-CoV-2 variants of concern: a systematic review and meta-analysis

Background It was urgent and necessary to synthesize the evidence for vaccine effectiveness (VE) against SARS-CoV-2 variants of concern (VOC). We conducted a systematic review and meta-analysis to provide a comprehensive overview of the effectiveness profile of COVID-19 vaccines against VOC. Methods Published randomized controlled trials (RCTs), cohort studies, and case-control studies that evaluated the VE against VOC (Alpha, Beta, Gamma, Delta, or Omicron) were searched until 4 March 2022. Pooled estimates and 95% confidence intervals (CIs) were calculated using random-effects meta-analysis. VE was defined as (1-estimate). Results Eleven RCTs (161,388 participants), 20 cohort studies (52,782,321 participants), and 26 case-control studies (2,584,732 cases) were included. Eleven COVID-19 vaccines (mRNA-1273, BNT162b2, ChAdOx1, Ad26.COV2.S, NVX-CoV2373, BBV152, CoronaVac, BBIBP-CorV, SCB-2019, CVnCoV, and HB02) were included in this analysis. Full vaccination was effective against Alpha, Beta, Gamma, Delta, and Omicron variants, with VE of 88.0% (95% CI, 83.0–91.5), 73.0% (95% CI, 64.3–79.5), 63.0% (95% CI, 47.9–73.7), 77.8% (95% CI, 72.7–82.0), and 55.9% (95% CI, 40.9–67.0), respectively. Booster vaccination was more effective against Delta and Omicron variants, with VE of 95.5% (95% CI, 94.2–96.5) and 80.8% (95% CI, 58.6–91.1), respectively. mRNA vaccines (mRNA-1273/BNT162b2) seemed to have higher VE against VOC over others; significant interactions (pinteraction < 0.10) were observed between VE and vaccine type (mRNA vaccines vs. not mRNA vaccines). Conclusions Full vaccination of COVID-19 vaccines is highly effective against Alpha variant, and moderate effective against Beta, Gamma, and Delta variants. Booster vaccination is more effective against Delta and Omicron variants. mRNA vaccines seem to have higher VE against Alpha, Beta, Gamma, and Delta variants over others. Supplementary Information The online version contains supplementary material available at 10.1186/s12916-022-02397-y.

Wuhan lineage virus [9]. The B.1.1.7 (Alpha) variant was first identified from genomic sequencing of samples obtained from COVID-19 patients which accounted for an expanding proportion of cases in England in late 2020 [10]. Subsequently, the emergence of the B.1.351 (Beta) variant in South Africa and the P.1 (Gamma) variant in Brazil increased the COVID-19 pandemic. In December 2020, a novel SARS-CoV-2 variant, the B.1.617.2 (Delta) variant was first detected in India, causing a sharp increase in COVID-19 cases and deaths in India and surrounding countries [11]. Recently, the B.1.1.529 (Omicron) variant emerged in December 2021 contains more than 30 mutations in the spike protein, raising concerns for naturally acquired or vaccinated population [12]. The emerging Alpha, Beta, Gamma, Delta, and Omicron variants were classified as variants of concern (VOC), which were associated with the transmission increasing, more severe disease situation (e.g., increased hospitalizations or deaths), significant reduction in neutralization by antibodies generated during previous infection or vaccination, reduced effectiveness of treatments or vaccines, or diagnostic detection failures [13][14][15][16][17][18]. The importance of vaccination programs and efficient public health measures will be increased if VOC have increased transmissibility or virulence [19]. It was urgent and necessary to synthesize evidence of the vaccine effectiveness (VE) of COVID-19 vaccines against VOC. To our knowledge, there are some studies evaluating the VE of COVID-19 vaccines against VOC [20][21][22][23]. Some relevant systematic review or meta-analysis about COVID-19 vaccines against Delta variant have been published to date [24][25][26], which did not include many recent studies as the most recent retrieval date was October 2021. Therefore, to gain insight in the VE of COVID-19 vaccines against five kinds of VOC, we conducted a comprehensive systematic review and meta-analysis including both RCTs and observational studies. This review of the VE of COVID-19 vaccines against VOC will support global response on public health measures and vaccination programs timely and evidence based.

Data sources and searches
We conducted this systematic review according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines [27]; the protocol was registered on PROSPERO (CRD42021273986). We searched for literature published on PubMed, Embase, Cochrane Library, and the Clini calTr ials. gov website on or before 4 March 2022. Keywords including "COVID-19, " "SARS-CoV-2, " "vaccine, " and "variant" were used to search; the detailed search strategy was shown in the Additional file 1 (Appendix S1). Additionally, we identified references by searching the reference lists of included studies and relevant reviews.

Selection of studies
We included randomized controlled trials (RCTs), cohort studies, and case-control studies that evaluated the efficacy or effectiveness of COVID-19 vaccines against VOC including B.1.1.7 (Alpha), B.1.351 (Beta), P.1 (Gamma), B.1.617.2 (Delta), and B.1.1.529 (Omicron). Studies enrolling general population or special populations (e.g., healthcare workers) aged 12 years or older were included. For studies that only reported VE against SARS-CoV-2 infections (without subgroup analysis of VOC), but the specific VOC accounted for 50% or more among positive tests, they were also included in the analysis. We excluded study protocols, editorials, comments, reviews, news, case reports, conference abstracts, animal studies, in vivo experiments, and analysis of antibody neutralization. Searches were limited to English articles. The primary outcome was the VE of full vaccination against VOC; the studies which only reported the VE of partial vaccination were excluded.

Data extraction
Two authors reviewed titles and abstracts independently to identify eligible studies that met pre-specified inclusion criteria and extracted data. When consensus was lacking, a third reviewer was consulted. The journal name, study type, study location, vaccine information, number of participants, characteristics of subjects, and outcomes were extracted from eligible studies. We extracted SARS-CoV-2 infection information if results on both SARS-CoV-2 infection and symptomatic infection were reported. The adjusted VE or estimates of effect size (relative risks, incidence rate ratios, or odds ratios) with corresponding 95% confidence intervals (CIs) were extracted with priority. The risk of bias of RCTs was assessed using the Cochrane Collaboration's tool [28,29]. The risk of bias of cohort and case-control studies was assessed using the Newcastle-Ottawa scale (NOS) [30]. The NOS contains 8 categories relating to methodological quality, with a maximum of 9 points. A total score of 7-9 points is considered of good quality, while a score of 4-6 points of moderate quality, and a score of 1-3 points of low quality. Two investigators reviewed the studies and judged the risk of bias.

Statistical analysis
Pooled estimates and 95% CIs were calculated using Der-Simonian and Laird random-effects meta-analysis [31]. Summary VE was defined as (1-pooled estimate) ×100%. We performed subgroup analysis stratifying by study design, vaccine type, participant, and publication. P for the difference was calculated using random-effects metaregression, a difference between the estimates of these subgroups was considered significant if p interaction < 0.10 [32]. Statistical heterogeneity between the studies was assessed with the χ 2 test and the I 2 statistics. I 2 values of 25%, 50%, and 75% have been suggested to be indicators of low, moderate, and high heterogeneity, respectively [33]. All the analyses were performed with STATA 14.

Risk of Bias
All the RCTs were assessed as some concerns for overall risk-of-bias judgment. Fifteen of 20 cohort studies were judged as good quality, and the remaining 5 studies were moderate quality. For 26 case-control studies, 22 were considered as good quality and 4 were moderate quality. The detailed risk of bias assessment is available in Additional file 1 (Tables S1-S3).
The summary VE of full vaccination against Beta variant was 73.0% (95% CI, 64.3-79.5) ( Table 2). Subgroup analysis by study design showed that VE for Beta variant was 45.6% (95% CI, −5.1 to 71.8) in 4 RCTs and 77.9% (95% CI, 72.6-82.2) in 7 real-world settings (case-control studies) (p interaction = 0.067). The VE against Beta variant of real-world evidence seems to be higher than RCTs.  20:200 Abbreviations: VOC variants of concern, HCWs healthcare workers, TNCC test-negative case-control, LTCH long-term care homes, GP general population, RCT randomized controlled trial, GS genomic sequencing, N number of participants, PRO prospective, RETRO retrospective a Median age (interquartile range) b Cases c VOC were identified by genomic sequencing (GS) or variant circulation dominance   2) for inactivated vaccines in 6 study groups, and 78.7% (95% CI, 55.1-89.9) for protein subunit vaccine in 1 study group (p interaction < 0.001). An interaction (p interaction < 0.001) between VE and vaccine type (mRNA vaccines vs. not mRNA vaccines) was found; the VE for mRNA vaccines seemed to be higher than others. The summary VE of booster vaccination against the Delta variant was 95.5% (95% CI, 94.2-96.5).

Vaccine effectiveness of COVID-19 vaccines against B.1.1.529 (Omicron) variant
Three case-control studies had evaluated the VE of COVID-19 vaccines against the Omicron variant [69,74,84]. Two mRNA vaccines (mRNA-1273 and BNT162b2) were included in this analysis. All three studies enrolled the general population. Characteristics of individual studies and VE for Omicron variant are summarized in Fig. 4 and Additional file 1 (Table S7). The summary VE of full vaccination against the Omicron variant was 55.9% (95% CI, 40.9-67.0), and the VE of booster vaccination against the Omicron variant was 80.8% (95% CI, 58.6-91.1).

Discussion
The VOC have mutations in its spike protein; most breakthrough cases were caused by contemporary variant strains [36]. The VE of current COVID-19 vaccines against VOC is concerning; we conducted this systematic review and meta-analysis to synthesize evidence on this topic during the pandemic. This study has five main findings. First, full vaccination of COVID-19 vaccines was effective against Alpha, Beta, Gamma, Delta, and Omicron variants, with the VE of 88.0%, 73.0%, 63.0%, 77.8%, and 55.9%, respectively. Second, booster vaccination Abbreviations: VE vaccine effectiveness, HCWs healthcare workers, LTCH long-term care homes, GP general population, RCT randomized controlled trial, ES effect size a P for interaction between vaccine effectiveness and vaccine type (mRNA vaccines vs. not mRNA vaccines)  (BNT162b2 or mRNA-1273). WHO guidelines recommend a lower bound of at least 30% and a vaccine efficacy of at least 50% [85]. The summary VE against Omicron variant was 55.9% of full vaccination and 80.8% of booster vaccination, raising concern for other vaccines. One study showed that Omicron variant extensively but incompletely escaped BNT162b2 neutralization [86]. Owing to multiple spike mutations, over 85% of tested neutralizing antibodies were escaped by Omicron variant, presenting a serious threat to existing therapies and COVID-19 vaccines [12,87].
The main results in this study were in consistent with a recent meta-analysis for neutralizing antibodies against SARS-CoV-2 variants, which showed that Alpha, Beta, Gamma, and Delta variants significantly escaped natural-infection-mediated neutralization, with an average of 1.4-fold, 4.1-fold, 1.8-fold, and 3.2-fold reduction in live virus neutralization assay [88]. Despite the reduction in neutralization titers against Alpha variant, they remain robust, and there is no evidence of vaccine escape in one study [89]. Escape of Beta variant from neutralization by convalescent plasma and vaccine-induced sera was observed in some studies [13,90,91]. Although neutralization titers against Gamma variant are reduced, it is hoped that immunization with vaccines designed against parent strains will protect Gamma variant infection [92]. The Delta variant escapes neutralization by some antibodies that target the receptor-binding domain or N-terminal domain; the neutralization titers against Delta were three to fivefolds than Alpha variant when two-dose of the vaccine administrated [15]. This study also supports the two-dose vaccine regimen recommended by the FDA and EMA, which is consistent with an in vitro study for SARS-CoV-2 variants of concern [93]. Also, booster vaccination demonstrates high VE against Delta infection in our study.
We did not evaluate VE against asymptomatic infection due to poor reporting in included studies. The summary VE against asymptomatic infection was slightly higher than any infection for Alpha, Gamma, and Delta variants, which was consistent with primary studies. The summary VE was higher against any infection with the Beta variant, which was probably confounded by study design and vaccine type. Three of 4 RCTs used symptomatic infection as an outcome, but 5 of 6 casecontrol studies used any infection. Most studies enrolled general population; only a few studies analyzed the VE in older adults or adolescents. We have performed subgroup analyses for VE against Delta variant stratifying by participants; the VE was 59.7% in older adults which was lower than general population (75.5%), healthcare workers (76.4%), and adolescents (88.8%). Vaccine type may be a confounder for this analysis, because one study showed that the VE of Ad26.COV2.S was much lower than BNT162b2 and mRNA-1273 in adolescents [63]. More evidence is needed for evaluating the VE against VOC in special population. This review included 3 study designs evaluating 11 COVID-19 vaccines against 5 VOC in different populations. There is high heterogeneity between studies, and high statistical heterogeneity is also observed in most analysis. Other factors like the definition of outcomes (all SARS-CoV-2 or symptomatic infection), days after vaccination, and participant's characteristics (e.g., age and race) may also contribute to the heterogeneity. Therefore, we mainly performed narrative descriptive synthesis.
This study has some limitations. First, 19% of studies (11 of 57) are nonrandomized. The imbalance between groups in observational studies is a concern, so potential selection bias may be existent. Second, we did not evaluate VE against asymptomatic infection due to poor reporting in included studies. Third, although we performed qualitative analysis by different stratifications, heterogeneity was still high in most quantitative analysis. Fourth, VE against hospitalization or death related to VOC is not included in our analysis. Finally, the evidence of COVID-19 vaccines against Omicron variant is not enough, more research is needed in the future.

Conclusions
Full vaccination of COVID-19 vaccines is highly effective against Alpha variant and moderate effective against Beta, Gamma, and Delta variant. Booster vaccination has more effectiveness against Delta and Omicron variants. mRNA vaccines (BNT162b2 or mRNA-1273) seem to have higher VE against Alpha, Beta, Gamma, or Delta over other vaccines. SARS-CoV-2 Omicron is raising concern for vaccinated individuals, and more evidence is needed to evaluate the VE of COVID-19 vaccines against the Omicron variant.
Additional file 1: Supplementary Materials. Search strategy (Appendix S1). Flow chart of literature search and study selection ( Figure S1). Risk of bias for included randomized controlled trials ( Table S1). Risk of bias for included cohort studies ( Table S2). Risk of bias for included casecontrol studies ( Table S3). VE of COVID-19 vaccines against Alpha variant ( Table S4). VE of COVID-19 vaccines against Beta and Gamma variant ( Table S5). VE of COVID-19 vaccines against Delta variant ( Table S6). VE of COVID-19 vaccines against Omicron variant (Table S7).