2021 Article 2149
2021 Article 2149
2021 Article 2149
Abstract
Background: The BNT162b2 mRNA vaccine has been shown to be effective at preventing serious COVID-19 events
in clinical trials. There is less evidence on effectiveness in real-world settings, especially for older people. Here, we
aimed to estimate vaccine effectiveness in the context of the rapid NHS mass-vaccination programme in England,
exploiting age-based vaccination eligibility thresholds to minimise and correct for selection bias.
Methods: We studied 170,226 individuals between the ages of 80 and 83 years from community settings outside
care homes who received one dose of BNT162b2 mRNA between the 15 and 20 December 2020 and were
scheduled a second dose 21 days later. We matched these vaccine recipients to slightly younger (aged 76–79 years)
persons not yet eligible to receive the vaccine on gender, area of residence, area deprivation, health status, living
arrangements, acute illness, and history of seasonal flu vaccination. We compared their rates of COVID-19 positivity
and hospitalisation in the subsequent 45 days. We adjusted for the increasing concentration of COVID-19 positivity
in the control population caused by the requirement to have no COVID-19 symptoms prior to vaccination.
Results: Emergency hospital admissions were 51.0% (95% confidence interval 19.9 to 69.5%) lower and positive
COVID-19 tests were 55.2% (40.8 to 66.8%) lower for vaccinated individuals compared to matched controls 21 to 27
days after first vaccination. Emergency admissions were 75.6% (52.8 to 87.6%) lower, and positive COVID-19 tests
were 70.1% (55.1 to 80.1%) lower 35 to 41 days after first vaccination when 79% of participants had received a
second dose within 26 days of their first dose.
Conclusions: Receipt of the BNT162b2 mRNA vaccine is effective at reducing COVID-19 hospitalisations and
infections. The nationwide vaccination of older adults in England with the BNT162b2 mRNA vaccine reduced the
burden of COVID-19.
Keywords: COVID-19, Vaccines, Infections, Observational study
* Correspondence: thomas.mason@nhs.net
1
NHS England & NHS Improvement, Quarry House, Quarry Hill, Leeds, West
Yorkshire LS2 7UE, UK
Full list of author information is available at the end of the article
© The Author(s). 2021 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,
which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if
changes were made. The images or other third party material in this article are included in the article's Creative Commons
licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons
licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain
permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the
data made available in this article, unless otherwise stated in a credit line to the data.
Mason et al. BMC Medicine (2021) 19:275 Page 2 of 9
Fig. 1 Numbers of people in England who received their first COVID-19 vaccination dose between 8 December 2020 and 3 February 2021 by age
group. The cumulative totals are relative to estimates of eligible population based on extracts from the National Health Application and
Infrastructure Services (NHAIS) system as of the 15 November 2020. Prior to the 4 January 2021, all individuals received the BNT162b2 mRNA
vaccine, after which individuals were vaccinated with either the BNT162b2 mRNA or ChAdOx1 adenovirus vector vaccines
would artificially inflate estimated vaccine effectiveness. We matched vaccinated individuals to unique controls
To correct for this bias, we sequentially adjusted event without replacement. We assigned a control randomly
rates in the intervention and control groups so that they where multiple matches were available for a vaccinated
remained consistent in the first 11 days of follow-up. individual. We repeated the matching process five times
Appendix 2 documents the adjustment method used. with different random number seeds to create five
matched populations. We bootstrapped 100 samples
with replacement from each of these five matched popu-
Matching and statistical analysis lations and obtained confidence intervals using percent-
Isolating the impact of vaccination requires a study ile values from the 500 samples.
design that accounts for temporal changes in infection
rates, for which we used 1:1 exact matching [13] to Tests of robustness
account for several factors associated with exposure To explore the robustness of the adjustment for selec-
and outcomes: gender; area of residence [14]; small tion bias in the control group and the sensitivity of the
area deprivation [15]; ethnic group; health status; liv- results to the age group used for the controls, we com-
ing arrangements; seasonal influenza vaccine history pared the older age group to two different younger age
since April 2020; and emergency hospital stays in the groups. First, we matched vaccinated individuals aged
previous 2 months. We excluded 1705 (1.0%) individ- 80–81 to controls aged 76–77 and vaccinated individuals
uals with prior COVID-19 history to avoid likely pre- aged 82–83 to controls aged 78–79. Second, we matched
existing immunity [16, 17]. We excluded individuals vaccinated individuals aged 80–81 to controls aged 72–
from the control group if they were living in care 73 and vaccinated individuals aged 82–83 to controls
homes or were not alive on the 15 December 2020. aged 74–75. The younger control group is less similar in
We dropped matched pairs where either individual age but unexposed to the vaccine for longer and there-
was in hospital on the vaccination date or the pair fore less prone to selection bias. We also tested the sen-
lived at the same property (see Fig. 2). sitivity of the results to reducing the inclusion criteria
Mason et al. BMC Medicine (2021) 19:275 Page 4 of 9
Fig. 2 Flow diagram of the study population with eligibility criteria, exclusions, and matching methodology
Mason et al. BMC Medicine (2021) 19:275 Page 5 of 9
for eligible controls from 2 to 1-week post first vac- was 55.2% (95% CI 40.8–66.8%) for documented infec-
cine dose (see Appendix 6). We used the STROBE tion, 57.8% (30.8–74.5%) for emergency hospital atten-
(Strengthening The Reporting of OBservational Stud- dances, and 50.1% (19.9–69.5%) for admissions. By days
ies in Epidemiology) cohort checklist when writing 35–41, the estimated effectiveness was 70.1% (55.1–
our report [18]. 80.1%) for documented infection, 78.9% (60.0–89.9%) for
emergency department attendances, and 75.6% (52.8–
Results 87.6%) for hospitalisations.
Study population
Of the total 1,685,530 individuals aged 80–83, 170,226 Discussion
met our inclusion criteria. They were not residents of We considered 171,931 individuals aged 80 to 83 years
care homes, had no prior history of COVID-19, and re- in England who received a first dose of BNT162b2
ceived a first dose of the BNT162b2 mRNA vaccine be- mRNA COVID-19 vaccine as part of the nationwide
tween 15 and 20 December 2020 (Fig. 2). Of these, we NHS vaccination campaign in England. We compared
exact-matched 131,236 (77.1%) to control individuals their rates of SARS-CoV-2 positive tests and COVID-19
aged 76–79 who were not yet eligible for vaccination hospitalisations in the subsequent 45 days to those for
(Figs. 1 and 2). The requirement for an exact match gen- slightly younger individuals with the same characteristics
erated a matched study population with lower propor- who became eligible for vaccination later. Emergency ad-
tions of individuals who were frail or clinically extremely mission was 50.1% (19.9 to 69.5%) less likely 21 to 27
vulnerable, from minority ethnic groups, or from days after vaccination and 75.6% (52.8 to 87.6%) less
socially-deprived areas compared to the full study popu- likely 35 to 41 days after first vaccination and 7 days after
lation (Table 1 and Appendix 4). 80% had received their second dose. COVID-19 infec-
tion was 55.2% (40.8 to 66.8%) less likely 21 to 27 days
Vaccine effectiveness after vaccination and 70.1% (55.1 to 80.1%) less likely 35
Across 45 days of follow-up, there was an average of to 41 days after first vaccination and 7 days after 80%
13.7 documented SARS-CoV-2 infections per day per had received their second dose. Collectively, these results
100,000 vaccinated individuals, compared to 23.2 per are consistent with one dose of the BNT162b2 mRNA
100,000 unvaccinated controls. Over the same period, a vaccine reducing events from 14 days after vaccination,
daily average of 5.0 individuals per 100,000 attended an with more effectiveness in reducing the severity of symp-
emergency department with COVID-19 and 5.3 per toms than preventing infection.
100,000 were hospitalised with COVID-19 amongst the Our results are broadly consistent with existing esti-
vaccinated cohort, compared to 9.6 per 100,000 mates of BNT162b vaccine effectiveness, despite varia-
(attended) and 9.4 per 100,000 (hospitalised) amongst tions in study design, participant demographics, and
unvaccinated controls. outcome definitions [21]. We estimate effectiveness
For the unvaccinated comparison group, COVID-19 against documented infection of approximately 55% 21–
event rates increased in the first 2 weeks of follow-up, 27 days after one dose, rising to 70% after the majority
with documented infections reaching a maximum at day received a second dose. These estimates are relatively
20, and emergency hospital attendances and admissions consistent with results from a similar studies in England
peaking between days 23 and 26 (Fig. 3). These profiles (55% after one dose, 80% 7 days after all received a sec-
reflect the shape of the COVID-19 pandemic in England ond dose) [8] and Scotland (78% 21–27 days after one
where prevalence peaked around the 1 January 2021 dose) [7] and from the older age group in a similar study
[19], and hospitalisations peaked in the second week of in Israel (50% after one dose, 95% 7 days after all re-
January 2021 [20]. For the vaccinated group, docu- ceived a second dose) [5]. These estimates are also com-
mented infections peaked earlier (day 14) and hospitali- parable to estimates of vaccine effectiveness against
sations peaked between days 23 and 26. documented infection in working-age adults in England
We found similar results when we matched vaccinated (70% after one dose, 85% 7 days after second dose) [9]
individuals to unvaccinated individuals aged 72–75 years and to all-age results from a randomised controlled trial
and when the comparison group was restricted to indi- (52% rising to 95%) [3]. Our point estimate of effective-
viduals who remained unvaccinated throughout the ness against hospitalisation with COVID-19 (76% 7 days
follow-up period (see Appendix 6). after most received a second dose) is somewhat lower
Table 2 shows vaccine effectiveness, defined as per- than, though statistically compatible with, other esti-
centage difference between vaccinated and unvaccinated mates (80–87%) [5, 8].
groups, for each outcome across four time periods. Ef- Several factors likely contribute to these differences.
fectiveness increased over the follow-up period for all First, our estimates are specifically for an older popula-
three outcomes. Estimated effectiveness at 21–27 days tion where vaccine-induced immune responses may be
Mason et al. BMC Medicine (2021) 19:275 Page 6 of 9
Table 1 Demographic and clinical characteristics of vaccinated persons and their unvaccinated controls based on the matched
cohort at baseline (day 11 after vaccination)
BAME Black, Asian and Minority Ethnicity, IMD Index of Multiple Deprivation, FY2020/2021 Financial Year 2020/2021, running from 1 April 2020 to 31 March 2021
sub-optimal [12]. In addition, our population included Finally, an important consideration in observational
20% of vaccinated individuals for whom the second dose studies is bias in selection into the intervention group.
was extended beyond the study period. This may explain While all existing studies used statistical methods to ad-
the greater agreement with existing estimates for effect- just for biases [5, 7, 8, 21], we exploited the precise age
iveness 21–27 days after vaccination than for longer thresholds that determined temporal eligibility for vac-
follow-up when second dose coverage varied between cination, thereby reducing the risk of unmeasured con-
studies. However, a study based in Scotland where the founding between cases and controls. Such biases are
majority received only single dose BNT162b estimated exacerbated with longer follow-up periods as those
87% (70 to 94) effectiveness against hospitalisation in remaining unvaccinated become increasingly different
those aged 80 and over at an equivalent time point (35– from those vaccinated earlier. The divergence between
41 days post vaccination) [7], suggesting that differences our effectiveness estimates and those in other studies
in second dose coverage may not explain the differences with longer follow-up may reflect less bias in our study
between estimated effectiveness. design and adjustment methodology.
Mason et al. BMC Medicine (2021) 19:275 Page 7 of 9
Fig. 3 Profiles of positive COVID-19 infections, emergency department (A&E) attendances, and unplanned hospital admissions by days since first dose
of vaccination. The data represent people aged between 80 to 83 years who received their first dose of the BNT162b2 mRNA COVID-19 vaccine
between the 15 and 20 December 2020 with comparison to their matched controls. 95% confidence intervals are displayed as dashed lines
We focused on older people at high risk of serious and area variables. We also compared four measures of
COVID-19 outcomes. We considered a period and hospital use in the previous 18 months and history of nega-
country experiencing widespread transmission and large tive SARS-CoV-2 tests (see Appendix 5). Vaccinated indi-
numbers of hospitalisations. This provided statistical viduals did not have lower event rates and had higher use
precision in the effectiveness estimates within a short of hospital services and more community-based COVID-19
period. We exploited a precise age cut-off that deter- tests prior to vaccination when compared to the control
mined access to the vaccine, which reduced bias from group. This likely reflects the age difference which may bias
selection into treatment. our estimates towards lower than true effectiveness.
Nonetheless, there is a risk of bias from unmeasured con- The rich set of matching variables meant some cases
founding with any observational study. We matched cases were excluded because there was no control available.
and controls on combinations of 12 personal, household These exclusions were more likely for some populations,
Table 2 Estimates of the effectiveness of the BNT162b2 mRNA COVID-19 vaccine by days since vaccination
Mason et al. BMC Medicine (2021) 19:275 Page 8 of 9
via the Policy Research Unit in Health and Social Care Systems and 6. Public Health England. Investigation of novel SARS-COV-2 variant: variant of
Commissioning. MS is a NIHR Senior Investigator. The views expressed are concern 202012/01. 2020 https://www.gov.uk/government/publications/
those of the authors and not necessarily those of the NIHR or the investigation-of-novel-sars-cov-2-variant-variant-of-concern-20201201.
Department of Health and Social Care. 7. Vasileiou E, Simpson CR, Shi T, et al. Interim findings from first-dose mass
COVID-19 vaccination roll-out and COVID-19 hospital admissions in
Scotland: a national prospective cohort study. Lancet. 2021;397(10285):
Availability of data and materials
1646–57.
Data used in the study are not publicly available and may be obtained via
8. Bernal JL, Andrews N, Gower C, et al. Early effectiveness of COVID-19
NHS Digital (https://digital.nhs.uk/services/data-access-request-service-dars).
vaccination with BNT162b2 mRNA vaccine and ChAdOx1 adenovirus vector
The analysis scripts used for the evaluation are available at https://github.
vaccine on symptomatic disease, hospitalisations and mortality in older
com/NHSEI-Analytics/nhs_covid19_effectiveness.
adults in the UK: a test negative case control study. medRxiv. 2021; https://
www.medrxiv.org/content/10.1101/2021.03.01.21252652v1.
Declarations 9. Hall VJ, Foulkes S, Saei A et al. COVID-19 vaccine coverage in health-care
workers in England and effectiveness of BNT162b2 mRNA vaccine against
Ethics approval and consent to participate infection (SIREN): a prospective, multicentre, cohort study. Lancet. 2021;
Surveillance of COVID-19 testing and vaccination is undertaken under Regu- 397(10286):1725–35.
lation 3 of The Health Service (Control of Patient Information) Regulations 10. Joint Committee on Vaccination and Immunisation. Advice on priority
2002 to collect confidential patient information (http://www.legislation.gov. groups for COVID-19 vaccination. 2020. https://www.gov.uk/government/
uk/uksi/2002/1438/regulation/3/made) under Sections 3(i) (a) to (c), 3(i)(d) (i) uploads/system/uploads/attachment_data/file/224864/JCVI_Code_of_Pra
and (ii) and 3(3). Review by the Heath Research Authority identified no regu- ctice_revision_2013_-_final.pdf (accessed March 11, 2021).
latory issues with this evaluation, and ethical review is not a requirement for 11. Williamson EJ, Walker AJ, Bhaskaran K, et al. Factors associated with COVID-
this study. 19-related death using OpenSAFELY. Nature. 2020;584:430–6.
Data analysis was facilitated under Control of patient information (COPI) 12. Boraschi D, Aguado MT, Dutel C, et al. The gracefully aging immune system.
notice (https://digital.nhs.uk/coronavirus/coronavirus-covid-19-response- Sci Transl Med. 2013;5:1–10.
information-governance-hub/control-of-patient-information-copi-notice). 13. Stuart EA. Matching methods for causal inference: a review and a look
Informed consent was not required for the study. forward. Stat Sci. 2010;25:1–21.
14. NHS. Middle Layer Super Output Area definition. NHS Data Model Dict.
https://datadictionary.nhs.uk/nhs_business_definitions/middle_layer_super_
Consent for publication output_area.html (accessed March 11, 2021).
The corresponding author has the right to grant on behalf of all authors and 15. Ministry of Housing Communities & Local Government. National Statistics:
does grant on behalf of all authors, a worldwide licence to the publishers English indices of deprivation 2019. 2019. https://www.gov.uk/government/
and its licensees in perpetuity, in all forms, formats, and media (whether statistics/english-indices-of-deprivation-2019 (accessed March 11, 2021).
known now or created in the future), to (i) publish, reproduce, distribute, 16. Grifoni A, Weiskopf D, Ramirez SI, et al. Targets of T cell responses to SARS-
display and store the contribution, (ii) translate the contribution into other CoV-2 coronavirus in humans with COVID-19 disease and unexposed
languages, create adaptations, reprints, include within collections and create individuals. Cell. 2020;181:1489–1501.e15.
summaries, extracts and/or, abstracts of the contribution, (iii) create any 17. Wajnberg A, Amanat F, Firpo A, et al. Robust neutralizing antibodies to
other derivative work(s) based on the contribution, (iv) to exploit all SARS-CoV-2 infection persist for months. Science (80- ). 2020;370:1227–30.
subsidiary rights in the contribution, (v) the inclusion of electronic links from 18. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke
the contribution to third party material where-ever it may be located; and JP. The Strengthening the Reporting of Observational Studies in
(vi) licence any third party to do any or all of the above. Epidemiology (STROBE) statement: guidelines for reporting observational
studies. J Clin Epidemiol. 2008;61:344–9.
Competing interests 19. Office of National Statistics (ONS). Coronavirus (COVID-19) Infection Survey,
All authors have completed the ICMJE (International Committee of Medical UK: 12 March 2021. 2021. https://www.ons.gov.uk/peoplepopulationa
Journal Editors) uniform disclosure form and declare the following: no ndcommunity/healthandsocialcare/conditionsanddiseases/bulletins/corona
support from any organisation for the submitted work, no financial viruscovid19infectionsurveypilot/11december2020#test-sensitivity-and-
relationships with any organisations that might have an interest in the specificity (accessed March 11, 2021).
submitted work in the previous 3 years, and no other relationships or 20. NHS England. COVID-19 Hospital Activity. 2021. https://www.england.nhs.uk/
activities that could appear to have influenced the submitted work. The statistics/statistical-work-areas/covid-19-hospital-activity/ (accessed March
authors declare that they have no competing interests. 11, 2021).
21. Kow CS, Hasan SS. Real-world effectiveness of BNT162b2 mRNA vaccine: a
Author details meta-analysis of large observational studies. Inflammopharmacology. 2021;
1 29:1075–90.
NHS England & NHS Improvement, Quarry House, Quarry Hill, Leeds, West
Yorkshire LS2 7UE, UK. 2Health Organisation, Policy & Economics, School of 22. Muik A, Wallisch A-K, Sanger B, et al. Neutralization of SARS-CoV-2 lineage B.
Health Sciences, University of Manchester, Manchester, UK. 1.1.7 pseudovirus by BNT162b2 vaccine-elicited human sera. Science (80- ).
2021;6105. https://doi.org/10.1126/science.abg6105.
Received: 28 April 2021 Accepted: 30 September 2021
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in
References published maps and institutional affiliations.
1. Baden LR, El Sahly HM, Essink B, et al. Efficacy and safety of the mRNA-1273
SARS-CoV-2 vaccine. N Engl J Med. 2021;384:403–16.
2. Voysey M, Clemens SAC, Madhi SA, et al. Safety and efficacy of the
ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim
analysis of four randomised controlled trials in Brazil, South Africa, and the
UK. Lancet. 2021;397:99–111.
3. Polack FP, Thomas SJ, Kitchin N, et al. Safety and efficacy of the BNT162b2
mRNA COVID-19 vaccine. N Engl J Med. 2020;383:2603–15.
4. Mahase E. COVID-19: Novavax vaccine efficacy is 86% against UK variant and
60% against South African variant. BMJ. 2021;372:2021.
5. Dagan N, Barda N, Kepten E, et al. BNT162b2 mRNA COVID-19 vaccine in a
nationwide mass vaccination setting. N Engl J Med. 2021;384:1412–23.