Nothing Special   »   [go: up one dir, main page]
Skip to main content
PMC home page
Scientific Reports logoLink to Scientific Reports
. 2021 Nov 23;11:22777. doi: 10.1038/s41598-021-02321-z

Comparing the clinical efficacy of COVID-19 vaccines: a systematic review and network meta-analysis

Victoria Rotshild 1,2,✉,#, Bruria Hirsh-Raccah 1,3,#, Ian Miskin 2, Mordechai Muszkat 4, Ilan Matok 1
PMCID: PMC8611039  PMID: 34815503

Abstract

New Coronavirus Disease 2019 (COVID-19) vaccines are available to prevent the ongoing severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic. We compared the efficacy of new COVID-19 vaccines to prevent symptomatic and severe disease in the adult population and to prevent symptomatic COVID-19 among the elderly. Leading medical databases were searched until August 30, 2021. Published phase 3 randomized controlled trials (RCTs) evaluated efficacy of the vaccine to prevent symptomatic and sever COVID-19 in adults were included. Two reviewers independently evaluated the literature search results and independently extracted summary data. The risk of bias was evaluated using the Cochrane Risk of Bias Assessment Tool. We performed a network meta-analysis (NMA) according to PRISMA-NMA 2015 to pool indirect comparisons between different vaccines regarding their relative efficacy. The primary outcomes were the efficacy of the vaccine against symptomatic COVID-19 in adults (PROSPERO registration number: CRD42021235364). Above 200,000 adult participants from eight phase 3 RCTs were included in NMA, of whom 52% received the intervention (active COVID-19 vaccine). While each of nine vaccines was tested in the unique clinical trial as compared to control, based on indirect comparison, BNT162b2 and mRNA-1273 vaccines were ranked with the highest probability of efficacy against symptomatic COVID-19 (P-scores 0.952 and 0.843, respectively), followed by Gam-COVID-Vac (P-score 0.782), NVX-CoV23730 (P-score 0.700), CoronaVac (P-score 0.570), BN02 (P-score 0.428), WIV04 (P-score 0.327), and Ad26.COV2.S (P-score 0.198). No statistically significant difference was seen in the ability of the vaccines to prevent symptomatic disease in the elderly population. No vaccine was statistically significantly associated with a decreased risk for severe COVID-19 than other vaccines, although mRNA-1273 and Gam-COVID-Vac have the highest P-scores (0.899 and 0.816, respectively), indicating greater protection against severe disease than other vaccines. In our indirect comparison, the BNT162b2 and mRNA-1273 vaccines, which use mRNA technology, were associated with the highest efficacy to prevent symptomatic COVID-19 compared to other vaccines. This finding may have importance when deciding which vaccine to use, together with other important factors as availability of the vaccines, costs, logistics, side effects, and patient acceptability.

Subject terms: Viral infection, Epidemiology

Introduction

In December 2019, a novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was first detected in Wuhan, China1. It causes highly infectious Coronavirus Disease 2019 (COVID-19) to spread worldwide and became a global pandemic. Despite numerous global efforts to mitigate the pandemic for almost two years, the SARS-CoV-2 continues to spread, disrupting life's routine, causing very high morbidity (above 225 million confirmed cases) and mortality (more than four and half million deaths) worldwide as of September 15, 20212.

Within a short period, it became clear that the way to deal with the current pandemic is an effective therapy for severe COVID-19 patients together with preventing SARS-Cov-2 spread through population vaccination. From the beginning of the pandemic, global efforts have been focused on developing safe and efficacious vaccines for COVID-19 prevention. Until recently, vaccine development was considered a long and complicated process, lasting for decades before the product has been approved for clinical use3. Shortly after the start of the SARS-Cov-2 outbreak, scientists began racing to develop an effective and safe vaccine against SARS-CoV-2, based on new and old vaccines technologies4.

Within less than two years period, there are more than 300 vaccine candidates globally, 117 vaccines in different clinical stages of development, including 30 of them in phase 35. As of mid-2021, seven COVID-19 vaccines have received emergency use authorization (EUA) in different countries, including United States (US), European Union (EU), United Kingdom (UK). These emergency authorizations of use are summarized in the World Health Organization (WHO) Emergency Use Listing: Pfizer/BioNTech (US, EU, UK, WHO), Moderna (US, EU, UK), AstraZeneca (EU, UK), Janssen (US, EU), and Gamaleya (Russian Ministry of Health), Sinopharm and Sinovac (National Medical Products Administration (NMPA), China)5.

The vaccines with EUA use various vaccine technologies, including mRNA6,7, virus vector8–10, and adjuvanted recombinant protein nanoparticles11. Each technology has its advantages and limitations12.

mRNA-12737 and BNT162b26 are the newest generations of mRNA vaccines. mRNA vaccines do not contain the antigen itself but deliver the genetic information for the antigen, and vaccinated individual synthesizes antigens in the host cells13. In this technology, all components are produced via chemical synthesis, which allows fast-track development in the event of a pandemic. The advantages associated with mRNA vaccines include high efficacy and relatively low severity of side effects. Before the current pandemic, mRNA vaccine technology seems promising in several diseases such as cytomegalovirus and Zika virus14, however, mRNA vaccines were not licensed for human use before the SARS-Cov-2 pandemic15. Thus, there are relatively short-term efficacy and safety data of COVID-19 mRNA vaccines, including recently published short-term real-world studies6,7,16–20. NVX-CoV2373 is an adjuvanted recombinant protein vaccine that contains Matrix-M1 adjuvant and a recombinant full-length wild-type SARS-CoV2 spike glycoprotein21. The same technology platform was used in the recently EU-approved Janssen Ebola vaccine22. ChAdOx1, Ad26CoV2.S, and Gam-COVID-Vac are viral vector-based vaccines8–10. The technology uses antigen cloned into a viral vector that cannot reproduce. The viral vector imitates the viral infection disease state and can produce more robust cellular immune responses compared to the recombinant protein vaccine. Adenoviral vector vaccines' safety has been extensively studied, and adenoviral vector-based therapeutic drugs are used in clinical practice23. In parallel with new technologies, recently published RCT reported the efficacy of three new whole-virus inactivated vaccines24,25.

For most new SARS-CoV-2 vaccines the efficacy data are based on the results of single phase 3 RCT, together with recently published real-world data for some of them6–11,17,19,24,25. Widespread vaccination programs have commenced in several countries, while the long-term effectiveness of COVID-19 vaccines is lacking. Recently published meta-analysis of eight COVID-19 vaccines, that have published the data of phase 3 randomized controlled trials (RCTs), reported excellent efficacy (pooled Risk Ratio (RR) to prevent symptomatic disease of 0.17; 95% Confidence Interval (CI): 0.09–0.32)26. While all new COVID-19 vaccines were found to be very effective to prevent symptomatic disease as compared to control, no study compared the efficacy between different vaccines.

The conventional meta-analysis approach can only compare two interventions at a time. Using the network methods enables the evaluation of multiple treatments in a single analysis. In the absence of a trial that directly compared two different treatments, an indirect comparison can be performed. Indirect evidence refers to the evidence obtained through a common comparator27. Network meta-analysis published on March 2021 included data about four COVID-19 vaccines and provided the following rank of effectiveness: BNT162b2 ≈ mRNA-1273 > Gam-COVID-Vac >  > ChAdOx128.

We aimed to integrate updated published data from phase 3 RCTs about different COVID-19 vaccines and provide an indirect comparison between vaccines' clinical efficacy to prevent symptomatic and severe disease, using network meta-analysis. Our results may provide additional evidence-based information to help choose the best policy to achieve the most significant public health benefit.

Methods

Data sources and search strategy

We performed a comprehensive database search which included PubMed/Medline, Embase, including Mesh/Emtree terms search, Clinical Trials Registry Clinicaltrials.gov, and The Cochrane Library using the following keywords: COVID-19, severe acute respiratory syndrome coronavirus, Coronaviridae Infections, coronavirus, sudden acute respiratory syndrome, vaccines, vaccine, randomized controlled trial, controlled clinical trial, clinical trial, phase II/III, phase III. The search strategies incorporated index terms (Mesh) and text words for the search concepts. The search words are detailed in online-only supplements. Databases were searched up to August 30, 2021, without language or date restrictions.

The primary outcomes were the clinical efficacy of the vaccine against symptomatic laboratory-confirmed COVID-19. Secondary outcomes were the efficacy to prevent severe COVID-19 infection and vaccine efficacy among the elderly.

The systematic review and network meta-analysis were performed following Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) 2020 framework guidelines29. The protocol was registered in the International Prospective Register of Systematic Reviews (PROSPERO) on February 5, 2021 (CRD42021235364).

Inclusion and exclusion criteria

We included published phase 3 RCTs to evaluate the vaccine's efficacy to prevent symptomatic COVID-19. The following publications were excluded from analysis: phase 1 and phase 2 RCTs, non-randomized trials, observational studies, duplicated reports, pharmacokinetic studies in healthy adults, reviews, expert opinion, editorials, letters to the editor, and comments.

Data extraction

One reviewer (V.R.) identified the studies. Two reviewers (V.R., B.H.R.) independently examined the list of titles, the abstracts, and finally, the full-text articles for eligibility using the Rayyan web software for systematic reviews30. Disagreements were resolved through consensus.

Data collection

The following data were extracted by two independent reviewers: study details (identifier, study design, geographical location, study period, publication year, length of follow up), participant details (number of participants, study population, age and gender, co-morbidities, SARS-Cov-2 variants), intervention details (vaccine name, vaccine platform, vaccine regimen), details about efficacy outcomes: number of cases of symptomatic disease, number of cases of severe disease, number of cases of symptomatic disease in participants above the age of 60 years (raw data). Disagreements between reviewers were resolved through consensus.

Quality assessment and risk of bias

The risk of bias of the randomized control trials was assessed by two independent reviewers using the Cochrane tool for assessing the risk of bias for randomized control trials (RCT)31.

Statistical analysis

We implemented a network meta-analysis according to PRISMA-NMA 201532. To investigate the differences in efficacy between various vaccines, we performed a pairwise network meta-analysis, using a random-effects model33–36. In the absence of trials that directly compared two different vaccines, only indirect comparisons have been performed. The network incorporated raw data of vaccine efficacy compared to control from each included study. RRs and 95% CIs for indirect comparisons between different vaccines regarding their relative efficacy was calculated using the pairwise method.

Vaccine efficacy was ranked using P-scores derived from network point estimates. The P-score is a frequentist equivalent to the Bayesian network surface under the cumulative ranking curve. The P-score of intervention can be interpreted as the mean extent of certainty that one intervention is better than another intervention, and can be used to rank an intervention within a range of interventions, measured on a scale from 0 (worst) to 1 (best)37.

To compare vaccines efficacy to prevent severe disease, we incorporated raw data of severe cases among vaccinated and control groups, as reported in each study. RRs and 95% CIs for indirect comparisons between different vaccines regarding their relative efficacy was calculated using the pairwise method. Vaccine’s efficacy to prevent severe disease was ranked using P-scores derived from network point estimates.

We applied pairwise network meta-analysis, using a random-effects model to compare vaccines’ efficacy to prevent symptomatic disease among the elderly. The network incorporated raw data of vaccine efficacy compared to control in patients above 60 years old from each included study. Vaccine’s efficacy to prevent symptomatic disease among the elderly was ranked using P-scores derived from network point estimates.

Analysis was performed using R Version 3.4.3 and the “netmeta” package Version 0.9–838.

Results

We identified eight phase-3 RCTs that reported primary or preliminary CODIV-19 vaccine efficacy, with contributory data from nine publications6–11,24,25,39.

The search and selection processes are illustrated in eFigure 1. The characteristics of included studies are summarized in Table 1. Data from above two hundred thousand participants are included in our network meta-analysis. Of whom 114,247 (52%) received the intervention (active COVID-19 vaccine), most of the participants (above 70%) are adults below the age of 60 years. The average number of participants per trial was 24,252 (± 9,877). A total of 1,419 cases of the primary outcome were reported in the included studies (eTable 1).

Table 1.

Characteristics of included studies.

Author Trial period Geographical location Intervention Vaccine type Pharma Regiment # participants Age (mean, range) Gender (male, %)
Polack FP6 July 27—Nov 14, 2020

US, Argentina, Brazil,

South Africa,

Germany, Turkey

 BNT162b2 mRNA

Pfizer/

BioNTech

2 doses,

21 days apart

 37,706

52

(16–91)

 50.6
Baden LR7 July 27—Oct 23, 2020 US mRNA-1273 mRNA Moderna

2 doses,

28 days apart

 30,351

51.4

(18–95)

 52.7
Voysey M8 April 23—Nov 4, 2020 UK, Brazil ChAdOx12

Viral Vector including

S-protein DNA

Astra Zeneca/

Oxford

2 doses,

4–12 weeks apart

 11,636 18 +  39.5
Logunov DY10 Sept 7—Nov 24, 2020 Russia Gam-COVID-Vac

Viral Vector including

S-protein cDNA

 Gamaleya NRCEM

2 doses,

21 days apart

 19,866

45

(SD 12)

 61.2
Heath PT 11 Sep 28 – Nov 28, 2020 UK NVX-CoV23730 Recombinant S-protein Novavax

2 doses

21 days apart

 14,039

56

(18–84)

 51.6
Sadoff J9 - January 22, 2021 US, South Africa, Latin America Ad26.COV2.S Viral vector expressing S protein

Janssen/

Johnsen & Johnsen

 1 dose 38,484

52.0

(18–100)

 54.9
Kaabi NA24 -December 20, 2020 United Arab Emirates, Bahrain WIV04, HB02 Inactivated viruse strains Sinopharm -Beijing 2 doses 21 days apart 38,206 36.1 (± 9.3) 84.4
Tanriover MD 25 Sept 15, 2020, and Jan 6, 2021 Turkey CoronaVac Inactivated whole-virion Sinovac Life Sciences 2 doses 21 days apart 10,214 18–59 57.8
Open in a new tab

Indirect comparison

Symptomatic disease

Our search revealed information about efficacy of nine new vaccines to prevent symptomatic COVID-19 (Table 1). When the indirect comparison between the vaccines was performed, BNT162b2 and mRNA-1273 vaccines were ranked with the highest probability of efficacy against symptomatic COVID-19 (P-score: 0.952, 0.843, respectively), followed by Gam-COVID-Vac (P-score 0.782), NVX-CoV23730 (P-score 0.700), CoronaVac (P-score 0.570), BN02 (P-score 0.428), WIV04 (P-score 0.327), ChAdOx1 (P-score 0.199), and Ad26.COV2.S (P-score 0. 0.198) (Table 2). BNT162b2, mRNA-1273, Gam-COVID-Vac, and NVX-CoV23730 vaccines were statistically significantly associated with a decreased risk for symptomatic COVID-19 (Fig. 1). Comparison of BNT162b2: RR 0.15, 95% CI: 0.07–0.31 vs. ChAdOx1 and Ad26.COV2.S; 0.23 (0.10–0.53) vs. HB02; 0.18 (0.08–0.42) vs. WIV04. Comparison of mRNA-1273: 0.21 (0.11–0.41) vs. ChAdOx1 and Ad26.COV2.S; 0.32 (0.15–0.70) vs. HB02; 0.26 (0.12–0.55) vs. WIV04. Comparison for Gam-COVID-Vac: 0.25 (0.14–0.46) vs. ChAdOx1 and Ad26.COV2.S; 0.38 (0.19–0.79) vs. HB02; 0.31 (0.15–0.62) vs. WIV04. Comparison for NVX-CoV23730: 0.31 (0.15–0.62) vs. ChAdOx1, and Ad26.COV2.S, and 0.38 (0.17–0.83) vs. WIV04.

Table 2.

P-Score ranking vaccines’ efficacy to prevent COVID-19.

Vaccine P-Score rankinga
Symptomatic disease Severe disease Symptomatic disease in elderlyb
BNT162b2 0.953 0.499 0.815
mRNA-1273 0.844 0.816 0.573
Gam-COVID-Vac 0.782 0.899 0.722
NVX-CoV2373 0.701 0.531 0.623
CoronaVac 0.570
HB02 0.428 0.384
WIV04 0.327 0.384
Ad26.COV2.S 0.198 0.434 0.262
ChAdOx1 0.199
Open in a new tab

aP-score represents the probability of each intervention is being better than all competing interventions, derived from network point estimates and standard errors.

bSubjects above 60 years.

Figure 1.

Figure 1

Open in a new tab

Results of random-effects network meta-analysis for efficacy to prevent symptomatic COVID-19: Risk Ratio (RR) for indirect comparison between the vaccines or vaccine vs. placebo, and 95% confidence intervals (Seven studies included).

Age 60 and above

Five studies reported vaccines' efficacy to prevent symptomatic disease among the older population (60 years and above) =6,7,9–11. The network incorporated 128 cases of symptomatic disease among patients above age 60 in vaccine and control groups, as reported in each study (eTable 1). When the indirect comparison between the vaccines was performed, BNT162b2 was ranked with the highest efficacy against symptomatic COVID-19 (P-score 0.815), followed by Gam-COVID-Vac (P-score 0.722), NVX-CoV23730 (P-score 0.623), mRNA-1273 (P-score 0.573), and Ad26.COV2.S (P-score 0.263) (Table 2). However, no vaccine was statistically significantly associated with a decreased risk compared to other vaccines (Fig. 2).

Figure 2.

Figure 2

Open in a new tab

Results of random-effects network meta-analysis for efficacy to prevent symptomatic COVID-19 in subjects ≥ 60 years old: Risk Ratio (RR) for indirect comparison between the vaccines or vaccine vs. placebo, and 95% confidence intervals (Four studies included).

Development of severe disease

Additionally, we evaluated the efficacy of the vaccines to prevent clinically significant severe COVID-19. The data of severe disease were available from five studies, a total of 107 cases of severe disease (eTable 1)6,7,9–11,24. eTable 2 summaraizes sever COVID-19 definitions, as defind in the inclided studies. When the indirect comparison between the seven vaccines was performed, Gam-COVID-Vac and mRNA-1273 vaccines were ranked with the highest efficacy to prevent a severe COVID-19 (P-scores 0.899 and 0.816, respectively), followed by NVX-CoV23730 (P-score 0.531), BNT162b2 (P-score 0.500), Ad26.COV2.S (P-score 0.34), WIV04 and HB02 (P-score 0.384) (Table 2). However, no vaccine was statistically significantly associated with a decreased risk compared to other vaccines, although there was a trend present with mRNA-1273 and Gam-COVID-Vac vaccines compared to the other vaccines for a lower risk for severe disease (Fig. 3).

Figure 3.

Figure 3

Open in a new tab

Results of random-effects network meta-analysis for efficacy to prevent severe COVID-19: Risk Ratio (RR) for indirect comparison between the vaccines or vaccine vs. placebo, and 95% confidence intervals (Five studies included).

Risk of bias

The risk of bias was evaluated for all published studies. It was classified as having some concerns for four studies6–8,11 and it was deemed moderate for other studies9,10,24,25 (eFigure 2).

Discussion

Over the last year, we have witnessed the development and clinical introduction of very effective COVID-19 vaccines, based on results from phase 3 RCTs. The two-dose regimen of BNT162b2 and mRNA-1273 mRNA, two vaccines based on new mRNA technology, presented extremely effective protection against COVID-19 (95% and 94.1%, respectively)6,7. Different regimens of viral-vector vaccines expressing SARC-CoV-2 S protein: Gam-COVID-Vac, Ad26.COV2.S, and ChAdOx1, were highly effective to protect against symptomatic COVID-19 (91.6%, 66.9%, and 66.7%, respectively)8–10. A two-dose regimen of the NVX-CoV2373, recombinant S-protein vaccine, administered to adult participants conferred 89.7% protection against SARS-CoV-2 infection11. Recently published results of three inactivated vaccines developed from different SARS-CoV-2 strains reported high efficacy for preventing COVID-19 symptomatic disease (83.5% CoronaVac, 78.1% HB02, and 72.8% WIV04)24,25. Combined data from phase 3 RCTs reported excellent efficacy of eight COVID-19 vaccines to prevent symptomatic disease as compared to control (RR 0.17; 95% CI 0.09–0.32)26.

The first network meta-analysis to compare the clinical efficacy of new COVID-19 vaccines was published on March 2021 and included four interventions: BNT162b2, mRNA-1273, Gam-COVID-Vac, and ChAdOx128. The current research is the most comprehensive network meta-analysis to compare the efficacy of nine new COVID-19 vaccines to prevent symptomatic and severe disease in the adult population.

Symptomatic disease

In our indirect comparison, the mRNA vaccines: BNT162b2 and mRNA-1273 were associated with the highest decrease in the relative risk for symptomatic COVID-19 compared to the other vaccines. BNT162b2 vaccine was associated with an 85% decreased relative risk of symptomatic disease than ChAdOx1 and Ad26.COV2.S (RR 0.15, 95% CI 0.07–0.31 and RR 0.15, 95% CI 0.07–0.31, respectively). the mRNA-1273 vaccine was 79% more effective in preventing symptomatic COVID-19 than ChAdOx1 and Ad26.COV2.S (RR 0.21, 95% CI 0.11–0.41 and RR 0.21, 95% CI 0.11–0.41, respectively) (Fig. 1). Ranking BNT162b2 and mRNA-1273 vaccines as best interventions over other competing vaccines to prevent symptomatic disease (P-score 0.95 and 0.84, respectively) (Table 2). Our results are consistent with previously published data, provided the following rank of effectiveness: BNT162b2 ≈ mRNA-1273 > Gam-COVID-Vac >  > ChAdOx128.

We did not find any statistically significant difference between the vaccines’ efficacy to prevent symptomatic disease among the elderly.

Development of severe disease

Among seven vaccines included in the analysis, Gam-COVID-Vac and mRNA-1273 vaccines were ranked with the highest probability to prevent a severe COVID-19 (P-scores 0.899 and 0.816, respectively) (Table 2). However, we did not find a statistically significant difference between the efficacy of Ad26.COV2.S vaccine to prevent severe COVID-19 as compared to Gam-COVID-Vac and mRNA-1273 vaccines (RR 0.11, 95% CI 0.01–2.15 for mRNA-1273 vs. Ad26.COV2.S and RR 0.05, 95% CI 0.01–1.05 for Gam-COVID-Vac vs. Ad26.COV2.S) (Fig. 2). We infer that there was not enough statistical power to compare vaccines’ efficacy to prevent severe COVID-19, as an absolute number of events was low (107 cases of severe COVID-19) (eTable 1).

The Ad26.COV2.S, ChAdOx1, and Gam-COVID-Vac are DNA vaccines encoding the SARS-CoV-2 spike (S) protein40. In our analysis, the Gam-COVID-Vac vaccine was more effective in preventing symptomatic COVID-19 as compared to Ad26.COV2.S and ChAdOx1 vaccines (RR 0.25, 95% CI 0.14–0.46 and RR 0.25, 95% CI 0.14–0.46, respectively) (Fig. 1). One possible explanation for the reduced efficacy of Ad26.COV2.S vaccine is a single-dose regimen compared to the two-dose regimen of Gam-COVID-Vac. A study is evaluating a two-dose administration of Ad26.COV2.S vaccine began participant recruitment during November 202141. Also, higher efficacy of Gam-COVID-Vac as indirectly compared to ChAdOx1 vaccine may be explained by two different vectors’ technology used in former. Using heterologous viral vectors for each dose allows the minimization of host immune responses against the vector components42. Three inactivated vaccines (HB02, WIV04, and CoronaVac) were developed from different SARS-CoV-2 strains isolated in China24,25. All three vaccines had comparable efficacy in preventing symptomatic COVID-19 (RR 0.81, 95% CI 0.43–1.54 for HB02 vs. VIW04, RR 1.49, 95% CI 0.62–3.57 for HB02 vs. CoronaVac, and RR 0.81, 95% CI 0.49–1.35 for VIW04 vs. CoronaVac) (Fig. 1).

Implications

Based on the indirect comparison method, the BNT162b2 and mRNA-1273 were associated with the highest efficacy in preventing symptomatic COVID-19. Our finding may have importance when deciding which vaccine to use, although this is not the only consideration that should be considered. Availability of the vaccines, costs, logistics, side effects, and patient acceptability, amongst others, are also factors to be considered.

Strengths and limitations

To the best of our knowledge, this is the most comprehensive network meta-analysis to compare the efficacy of nine new COVID-19 vaccines to prevent symptomatic and severe disease in the adult population. Previously published network meta-analysis reported indirect comparisons across fore COVID-19 vaccines28.

The results of our indirect comparison between the new vaccines showed that mRNA vaccines (BNT162b2 and mRNA-1273) were associated with a more significant decrease in the risk for symptomatic COVID-19 compared to other vaccines. We also found a trend to increased the efficacy of mRNA vaccines to prevent severe COVID-19. However, the results did not reach statistical significance because of the relatively low rate of severe disease.

However, our indirect comparison has several limitations.

Firstly, our network meta-analysis includes one study for each intervention arm. In addition, the results of the two studies are not peer-reviewed, while reported data originated from press releases and reports submitted to FDA43,44. There are several significant differences between studies' protocols, which may be partially responsible for the differences between the vaccine efficacies. As mentioned above, Ad26.COV2.S efficacy is based on a single-dose regimen, while other vaccines were administered as a two-dose regimen, including ChAdOx1 (AZD1222) vaccine, whose protocol was adapted to a two-dose regimen after the study had been started45. Moreover, vaccines were examined under non-equivalent conditions, including countries with unlike socio-economic conditions and various stages of COVID-19 outbreak, different seasons, and different SARS-CoV-2 variants. All mentioned above may influence vaccines' efficacy. Recently published data support that the B.1.1.17 variant, known as the UK strain, is susceptible to the immunity induced bytheBNT162b2 and mRNA-1273 vaccines46,47. However, the B.1.351 variant, primarily identified in South Africa, is less susceptible to mRNA-1273 vaccine-induced neutralizing antibodies47. It remains to determine if the reduction in antibody susceptibility will be associated with decreased vaccine effectiveness. There is also a high probability that the virus will acquire new mutations that will change its susceptibility to vaccines, and some vaccines might be influenced more than others. As a result, the efficacy of the different vaccines is expected to be affected. Besides, the current data on vaccine efficacy is based on short-term data, so we could not compare the effectiveness and immunity duration of different vaccines. Presently, it is not known which vaccine will induce longer immune responses. Also, as seen with other vaccines, booster doses may be required every few years to maintain immunity.

Secondly, our meta-analysis compares the efficacy of the studied vaccines in two hundred thousand participants of phase 3 RCTs without data from observational studies. So far, millions of people have been vaccinated around the world. One study from Clalit Health Services, a large health maintenance organization in Israel, compares the efficacy of the BNT162b2 mRNA vaccine in about 600,000 vaccinated persons to that of a similar-sized group of unvaccinated controls48. In this study, the efficacy of the BNT162b2 mRNA vaccine was similar to that seen in the phase 3 RCT7.

Finally, the safety outcomes of the vaccines were beyond the aims of the current network meta-analysis. Currently, available safety results are based on short-duration follow-up, and a very low rate of severe adverse reactions has been observed in the short term. As the mRNA vaccine technology is new and it is still unclear which issues will emerge in the long term, real-world data will be needed to assess the safety of prospective vaccines.

Conclusion

In our indirect comparison, the BNT162b2 and mRNA-1273 vaccines, which use mRNA technology, were associated with the highest efficacy in preventing symptomatic COVID-19 compared to the other vaccines. The compared vaccines were not different in efficacy to prevent severe disease. We found no difference between vaccines’ efficacy to prevent symptomatic COVID-19 among the elderly.

Supplementary Information

Supplementary Information. (114.5KB, docx)

Acknowledgements

The authors gratefully acknowledge Tomer Ben Shoshan for helping with the search strategy and the extraction of the records.

Author contributions

V.R. and B.H.R. contributed equally to the paper and share co-leading authorship. V.R. and B.H.R. contributed to the study design and data analysis and wrote the first draft of the manuscript. V.R. had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Ia.M. directed the research, contributed to the study design, analysis of data, interpretation of study results, and writing of the manuscript. I.M. contributed to the interpretation of the study results and to the drafting of the manuscript. M.M. provided critical appraisal and revision, and contributed substantially to the study interpretation. All authors reviewed the manuscript.

Funding

There was no funding source for this study.

Competing interests

The authors declare no competing interests.

Footnotes

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

These authors contributed equally: Victoria Rotshild and Bruria Hirsh-Raccah.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-021-02321-z.

References

  • 1.Zhu N, et al. A novel coronavirus from patients with pneumonia in China, 2019. N. Engl. J. Med. 2020;382:727–733. doi: 10.1056/NEJMoa2001017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.WHO Coronavirus (COVID-19) Dashboard | WHO Coronavirus (COVID-19) Dashboard With Vaccination Data. https://covid19.who.int/.
  • 3.Vaccine Development, Testing, and Regulation | History of Vaccines. https://www.historyofvaccines.org/content/articles/vaccine-development-testing-and-regulation.
  • 4.Papageorgiou AC, Mohsin I. The SARS-CoV-2 spike glycoprotein as a drug and vaccine target: structural insights into its complexes with ACE2 and antibodies. Cells. 2020;9:2. doi: 10.3390/cells9112343. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.COVID-19 vaccine tracker and landscape. https://www.who.int/publications/m/item/draft-landscape-of-covid-19-candidate-vaccines.
  • 6.Polack FP, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N. Engl. J. Med. 2020;383:2603–2615. doi: 10.1056/NEJMoa2034577. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Baden LR, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N. Engl. J. Med. 2021;384:403–416. doi: 10.1056/NEJMoa2035389. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Voysey M, et al. Single-dose administration and the influence of the timing of the booster dose on immunogenicity and efficacy of ChAdOx1 nCoV-19 (AZD1222) vaccine: a pooled analysis of four randomised trials. Lancet. 2021;397:881. doi: 10.1016/S0140-6736(21)00432-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Sadoff J, et al. Safety and efficacy of single-dose Ad2.6COV2.S vaccine against COVID-19. N. Engl. J. Med. 2021 doi: 10.1056/NEJMoa2101544. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Logunov DY, et al. Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia. Lancet. 2021;397:671–681. doi: 10.1016/S0140-6736(21)00234-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Heath PT, et al. Safety and efficacy of NVX-CoV2373 Covid-19 vaccine. N. Engl. J. Med. 2021 doi: 10.1056/NEJMOA2107659. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Awadasseid A, Wu Y, Tanaka Y, Zhang W. Current advances in the development of SARS-CoV-2 vaccines. Int. J. Biol. Sci. 2021;2021:8–19. doi: 10.7150/ijbs.52569. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Krammer F. SARS-CoV-2 vaccines in development. Nature. 2020;586:516–527. doi: 10.1038/s41586-020-2798-3. [DOI] [PubMed] [Google Scholar]
  • 14.Pardi N, et al. Zika virus protection by a single low-dose nucleoside-modified mRNA vaccination. Nature. 2017;543:248–251. doi: 10.1038/nature21428. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Xu, S., Yang, K., Li, R. & Zhang, L. Molecular sciences mRNA vaccine era-mechanisms, drug platform and clinical prospection. doi:10.3390/ijms21186582. [DOI] [PMC free article] [PubMed]
  • 16.Angel Y, et al. Association between vaccination with BNT162b2 and incidence of symptomatic and asymptomatic SARS-CoV-2 infections among health care workers. JAMA - J. Am. Med. Assoc. 2021 doi: 10.1001/jama.2021.7152. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Haas EJ, et al. Impact and effectiveness of mRNA BNT162b2 vaccine against SARS-CoV-2 infections and COVID-19 cases, hospitalisations, and deaths following a nationwide vaccination campaign in Israel: an observational study using national surveillance data. Lancet. 2021;397:1819–1829. doi: 10.1016/S0140-6736(21)00947-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Hall VJ, et al. COVID-19 vaccine coverage in health-care workers in England and effectiveness of BNT162b2 mRNA vaccine against infection (SIREN): a prospective, multicentre, cohort study. Lancet. 2021;397:1725–1735. doi: 10.1016/S0140-6736(21)00790-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Dagan N, et al. BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting. N. Engl. J. Med. 2021 doi: 10.1056/NEJMoa2101765. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Lopez Bernal J, et al. Effectiveness of the Pfizer-BioNTech and Oxford-AstraZeneca vaccines on covid-19 related symptoms, hospital admissions, and mortality in older adults in England: test negative case-control study. BMJ. 2021 doi: 10.1136/bmj.n1088. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Keech C, et al. Phase 1–2 trial of a SARS-CoV-2 recombinant spike protein nanoparticle vaccine. N. Engl. J. Med. 2020;383:2320–2332. doi: 10.1056/NEJMoa2026920. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Vaccine against Ebola. https://ec.europa.eu/commission/presscorner/detail/en/ip_20_1248.
  • 23.Wold W, Toth K. Adenovirus vectors for gene therapy, vaccination and cancer gene therapy. Curr. Gene Ther. 2014;13:421–433. doi: 10.2174/1566523213666131125095046. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Al Kaabi N, et al. Effect of 2 inactivated SARS-CoV-2 vaccines on symptomatic COVID-19 infection in adults: A randomized clinical trial. JAMA. 2021 doi: 10.1001/jama.2021.8565. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Tanriover MD, et al. Efficacy and safety of an inactivated whole-virion SARS-CoV-2 vaccine (CoronaVac): interim results of a double-blind, randomised, placebo-controlled, phase 3 trial in Turkey. Lancet. 2021;398:213–222. doi: 10.1016/S0140-6736(21)01429-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.H C, et al. Efficacy and safety of COVID-19 vaccines in phase III trials: a meta-analysis. Vaccines. 2021;9:2. doi: 10.3390/vaccines9060582. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Rouse B, Chaimani A, Li T. Network meta-analysis: an introduction for clinicians. Intern. Emerg. Med. 2021;12:2. doi: 10.1007/s11739-016-1583-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Calzetta L, et al. Factors influencing the efficacy of COVID-19 vaccines: a quantitative synthesis of phase III trials. Vaccines. 2021;9:341. doi: 10.3390/vaccines9040341. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Moher D, Liberati A, Tetzlaff J, Altman DG. Systematic reviews and meta-analyses CHECK LIST: The PRISMA statement. PLoS Med. 2009;6:e1000097. doi: 10.1371/journal.pmed.1000097. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst. Rev. 2016;5:1–10. doi: 10.1186/s13643-016-0384-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Higgins JPT, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:2. doi: 10.1136/bmj.d5928. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Hutton B, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162:777–784. doi: 10.7326/M14-2385. [DOI] [PubMed] [Google Scholar]
  • 33.Gorelik E, et al. Systematic review, meta-analysis, and network meta-analysis of the cardiovascular safety of macrolides. Antimicrob. Agents Chemother. 2018;62:2. doi: 10.1128/AAC.00438-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Gorelik E, et al. Fluoroquinolones and cardiovascular risk: a systematic review Meta-analysis and Network Meta-analysis. Drug Saf. 2019;42:2. doi: 10.1007/s40264-018-0751-2. [DOI] [PubMed] [Google Scholar]
  • 35.Leshem R, et al. Selective serotonin reuptake inhibitors (SSRIs) and serotonin norepinephrine reuptake inhibitors (SNRIs) during pregnancy and the risk for autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD) in the offspring: A true effect o. Curr. Neuropharmacol. 2021;19:896–906. doi: 10.2174/1570159X19666210303121059. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.R M, et al. Prenatal exposure to selective serotonin reuptake inhibitors and serotonin norepinephrine reuptake inhibitors and risk for persistent pulmonary hypertension of the newborn: a systematic review, meta-analysis, and network meta-analysis. Am. J. Obstet. Gynecol. 2019;220:57–57. doi: 10.1016/j.ajog.2018.08.030. [DOI] [PubMed] [Google Scholar]
  • 37.Rücker G, Schwarzer G. Ranking treatments in frequentist network meta-analysis works without resampling methods. Med. Res. Methodol. 2015;15:58. doi: 10.1186/s12874-015-0060-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Package ‘netmeta’ Title Network Meta-Analysis using Frequentist Methods. (2021) doi:10.1007/978-3-319-21416.
  • 39.Ramasamy MN, et al. Safety and immunogenicity of ChAdOx1 nCoV-19 vaccine administered in a prime-boost regimen in young and old adults (COV002): a single-blind, randomised, controlled, phase 2/3 trial. Lancet. 2020;396:1979–1993. doi: 10.1016/S0140-6736(20)32466-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Bos R, et al. Ad26 vector-based COVID-19 vaccine encoding a prefusion-stabilized SARS-CoV-2 Spike immunogen induces potent humoral and cellular immune responses. NPJ Vaccines. 2020;5:1–11. doi: 10.1038/s41541-020-00243-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Janssen Vaccines & Prevention B.V. A Study of Ad26.COV2.S in Adults (COVID-19). https://www.clinicaltrials.gov/ct2/show/NCT***04436276 (2020).
  • 42.Lu, S. Heterologous prime-boost vaccination. doi:10.1016/j.coi.2009.05.016. [DOI] [PMC free article] [PubMed]
  • 43.Novavax COVID-19 Vaccine Demonstrates 89 . 3 % Efficacy in UK Phase 3 Trial. (2021).
  • 44.Biotech, J. Vaccines and Related Biological Products Advisory Committee February 26, 2021 Meeting Briefing Document- FDA. (2021).
  • 45.Voysey, M. et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. www.thelancet.com397, 2021 (2020). [DOI] [PMC free article] [PubMed]
  • 46.Xie X, et al. Neutralization of N501Y mutant SARS-CoV-2 by BNT162b2 vaccine-elicited sera. Biorxiv Prepr Serv. Biol. 2021 doi: 10.1101/2021.01.07.425740. [DOI] [Google Scholar]
  • 47.Wu, K. et al. mRNA-1273 vaccine induces neutralizing antibodies against spike mutants from global SARS-CoV-2 variants. bioRxiv Prepr. Serv. Biol. 2021.01.25.427948 (2021) doi:10.1101/2021.01.25.427948.
  • 48.Noa D, Noam B, Eldad K, Shay P, Katz MA, Hernán MA, Lipsitch M, Ben R, Balicer RD. BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting. N. Engl. J. Med. 2021 doi: 10.1056/NEJMoa2101765. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Information. (114.5KB, docx)

Articles from Scientific Reports are provided here courtesy of Nature Publishing Group

RESOURCES