Abstract
Background
Understanding the threat posed by anti-vaccination efforts on social media is critically important with the forth coming need for world wide COVID-19 vaccination programs. We globally evaluate the effect of social media and online foreign disinformation campaigns on vaccination rates and attitudes towards vaccine safety.
Methods
We use a large-n cross-country regression framework to evaluate the effect of social media on vaccine hesitancy globally. To do so, we operationalize social media usage in two dimensions: the use of it by the public to organize action (using Digital Society Project indicators), and the level of negatively oriented discourse about vaccines on social media (using a data set of all geocoded tweets in the world from 2018-2019). In addition, we measure the level of foreign-sourced coordinated disinformation operations on social media ineach country (using Digital Society Project indicators). The outcome of vaccine hesitancy is measured in two ways. First, we use polls of what proportion of the public per country feels vaccines are unsafe (using Wellcome Global Monitor indicators for 137 countries). Second, we use annual data of actual vaccination rates from the WHO for 166 countries.
Results
We found the use of social media to organise offline action to be highly predictive of the belief that vaccinations are unsafe, with such beliefs mounting as more organisation occurs on social media. In addition, the prevalence of foreign disinformation is highly statistically and substantively significant in predicting a drop in mean vaccination coverage over time. A 1-point shift upwards in the 5-point disinformation scale is associated with a 2-percentage point drop in mean vaccination coverage year over year. We also found support for the connection of foreign disinformation with negative social media activity about vaccination. The substantive effect of foreign disinformation is to increase the number of negative vaccine tweets by 15% for the median country.
Conclusion
There is a significant relationship between organisation on social media and public doubts of vaccine safety. In addition, there is a substantial relationship between foreign disinformation campaigns and declining vaccination coverage.
Keywords: vaccines
Key questions.
What is already known?
Vaccine hesitant groups on social media have an alarming foot print, with studies demonstrating that large proportions of the content about vaccines on popular social media sites are anti-vaccination messages.
Organized campaigns have been traced to Russian pseudo-state actors promoting anti-vaccination content on social media abroad.
What are the new findings?
At a national level, the use of social media to organize offline action is highly predictive of the belief that vaccinations are unsafe, with such beliefs mounting as more organization occurson social media.
Foreign disinformation campaigns online are associated with a drop in both mean vaccination coverage over time and negative discussion of vaccines on social media.
A one-point shift upwards in the five-point disinformation scale is associated with a two-percentage point drop in mean vaccination coverage year over year and a 15% increase in negative tweets about vaccines.
What do the new findings imply?
The need for coordinated action removing anti-vaccination content from social media platforms.
The need for coordinated action against the sources of intentional anti-vaccination disinformation campaigns abroad.
Introduction
Last year the WHO listed vaccine hesitancy as one of the top 10 threats to world health. Historically, democracies have been associated with improved health outcomes due to institutions being accountable to the public, increased levels of public education and generally higher levels of wealth. Paradoxically though, contemporary anti-vaccination sentiment appears to be most concentrated in wealthy and highly educated democracies. Social media, while providing an unprecedented capacity for the public to communicate, has also been a major factor in the rise of fringe opinions damaging to public health. Reconciling principles of free speech with the policing of social media for damaging falsehoods remains a conundrum for democracies.
Vaccine hesitancy is not a new phenomenon, but the proliferation of anti-vaccination misinformation through social media has given it new urgency, especially in light of the coronavirus pandemic and hopes for rapid development and deployment of a vaccine. While there has been a great deal of work on the socioeconomic determinants of vaccine hesitancy ranging from qualitative single-country work to large scale surveys across dozens of countries, there has not been a global cross-national analysis of the effect of social media.1–8 This article fills that gap by addressing a pair of research questions tackling two dimensions of the proposed effect of social media.
Vaccine hesitant groups on social media have an alarming footprint, with studies from the early 2000s to the present showing that large proportions of the content about vaccines on popular social media sites are anti-vaccination messages.9–17 By drastically reducing the transaction costs associated with communication, social media has a 'long tail' effect in which the lack of any barrier to entry allows fringe groups to broadcast their message. In the case of anti-vaccination messaging, an effect similar to that of ethnic outbidding can emerge18 in which a fringe group’s misinformation gains traction, not because it is considered credible but because, on the unlikely chance it is correct, the consequences would be horrific. More extreme propaganda of negative effects is incentivised, thus leading to a spiral of threat matched by public fear. This leads to our first research question: does social media usage increase vaccine hesitancy in populations?
However, the effect of social media is compounded by an additional factor: the intentional spread of disinformation in addition to misinformation. Research has shown that Russian bots and troll farms, in conjunction with Russia’s foreign broadcast network RT, have pushed anti-vaccination messages on a large scale on Western social media.19 20 These messages may be part of a broader effort to strategically undermine public health in both the developed and developing world. For example, in the Democratic Republic of the Congo during both the 2014 and 2019 Ebola outbreaks, disinformation campaigns argued that foreign medical workers were spreading the disease, contributing to attacks that killed or wounded dozens of medical workers.21 And most recently, coronavirus disinformation arguing alternately that it was an American-developed weapon or caused by 5G cellular networks has also been pushed by Russian bot networks.22 This ongoing campaign serves Russian strategic interests of mitigating American influence abroad, even at the cost of health outcomes. This leads to our second research question: do foreign disinformation campaigns increase vaccine hesitancy in populations?
Methods
In 2016 Thomson et al proposed a taxonomy of '5-As' to describe the dimensions of vaccine uptake: access, affordability, awareness, acceptance and activation.23 We use that framework as a starting point for understanding how to model the effect of social media on vaccine hesitancy. Misinformation by vaccine hesitant groups on social media about the safety of vaccines explicitly targets acceptance. In addition, activation is defined as attempts to actively remind individuals and prod them into getting vaccines, and foreign disinformation attacks this directly by encouraging the opposite. The dimensions of access, affordability and awareness have received most of the previous attention in the literature, and we address them with a suite of control variables discussed later.
We use a cross-country regression framework to evaluate the effect of social media on vaccine hesitancy globally. To do so, we operationalise social media usage in two dimensions: the use of it by the public to organise action, and the level of negatively orientated discourse about vaccines on social media. In addition, we independently measure the level of foreign-sourced coordinated disinformation operations on social media. The outcome of vaccine hesitancy is measured in two ways. First, we use polls of the proportion of the public per country who feel vaccines are unsafe. Second, we use annual data of actual vaccination rates. In this section we will first address our measurement strategies and then discuss our regression design.
Measurement
Measuring negative discussion of vaccines
Approximately 1.5% of tweets worldwide are geocoded, with an attached place generated either from contextual clues or the GPS of the device. Using an existing database of all geocoded tweets in the world from 2018 to 2019 that identifies country of origin, we created a measure of the overall level of Twitter usage per country on the basis of some 2.5 billion tweets.24
We further extracted from these data the subset of tweets that mentioned a simple set of vaccine-related terms ('vaccine', 'vaccination', 'vaxx') with all grammatical variants in each of the top 20 languages used on Twitter (which account for over 95% of tweets).25 This amounted to 258 769 vaccination-related tweets.
While geocoded tweets represent a small proportion of tweets overall, they have some very specific advantages in this context. First, they allow us to identify country of origin for our cross-national analysis. Second, geocoded tweets do not include retweets (which represent 80% of tweets overall), which means that each tweet represents content actually entered by the individual in question as opposed to simply being passed along. Third, geocoded tweets have far lower rates of bot activity, which allows us more confidence that this is a measure of public attitude as opposed to the prevalence of anti-vaccination bot activity.
We measured the sentiment of each vaccine-related tweet using the Polyglot Python library, which has sentiment lexicons labelling words as positive, negative or neutral in 136 languages.26 27 We used Twitter’s machine-identified language for each tweet to select the appropriate sentiment lexicon and classified tweets as negative if their total negative words exceeded their positive words. We aggregated the total number of tweets and total number of negative vaccine-related tweets to the country level.
Public use of social media to organise
In order to measure how much the public uses social media to organise action of any kind, we use the 'average people’s use of social media to organise offline action' variable (v2smorgavgact) from the Digital Society Project (DSP) dataset.28 The DSP data is a set of 35 indicators focused on the intersection of the internet and politics, constructed using the expert coding methodology and infrastructure of the Varieties of Democracy (V-Dem) Institute.29
This measurement approach is based on the principle that many social and political qualities of countries are not directly measurable (unlike many economic factors such as income), but are still objective facts that can be readily quantified and described by experts on the topic and country. For instance, to what degree citizens in practice are likely to be arrested for criticising the government is a factual question that experts on politics in a particular country can answer. V-Dem has recruited a network of more than 3000 scholars from over 180 countries, three-quarters of whom have doctoral or equivalent degrees in a relevant field. These experts are annually asked a suite of Likert scale questions (multiple choice questions in which each choice has a description rather than arbitrary numerical weighting) on the topic of their expertise. These responses are aggregated with a Bayesian measurement model that takes into account cross-country comparability, intercoder reliability and anchoring vignettes to establish individual thresholds.30 The DSP indicators focus on the internet and social media in a time series extending from 2000 to the present, in all countries. The survey was responded to by media scholars in each country.
This particular indicator regarding social media usage for organisation asks: “How often do average people use social media to organise offline political action of any kind?” on a 5-point Likert scale ranging from “Never or almost never” to “Regularly. There are numerous cases in which average people have used social media to organise offline political action”.
Foreign disinformation
We again make use of the DSP dataset to measure the preponderance of foreign-sourced disinformation on domestic social media in each country using the 'foreign governments dissemination of false information' variable (v2smfordom).28
This indicator asks: "How routinely do foreign governments and their agents use social media to disseminate misleading viewpoints or false information to influence domestic politics in this country?” on a 5-point Likert scale ranging from “Extremely often. Foreign governments disseminate false information on all key political issues” to "Never, or almost never”. For the purposes of this paper, we invert the measurement scale so that a higher value indicates more disinformation rather than less.
Public attitudes towards vaccination
In order to measure national variance in attitudes towards vaccination, we draw on the 2019 Wellcome Global Monitor which surveyed individuals in 140 countries on a variety of science and health issues.31 We use the national level results for question 25 of the survey, which asks respondents whether they “strongly or somewhat agree, strongly or somewhat disagree or neither agree nor disagree” with the statement “vaccines are safe”.
We use the sum of the “strongly disagree” and “somewhat disagree” categories to measure the percentage of public doubt of safety of vaccines. In addition, we use the crosstab results per country that break down each response by socioeconomic categories: gender, age (15–29, 30–49, 50+), level of education (less than secondary, secondary, tertiary), urban versus rural, and income level (categorised by quintile).
Vaccination rates
The WHO tracks reported estimates of vaccination coverage for 22 vaccine packages at a country-year level.32 Because there is significant variation in which vaccines are given in which countries, we generated an aggregate 'mean vaccination rate'.
We identified the top 10 most commonly reported vaccine doses (DTP3, MCV1, Pol3, Hib3, DTP1, HepB3, IPV1, BCG, MCV2 and RCV1) (see table 1) in order to avoid skewing the aggregate with uncommon and non-representative reporting. The number of countries reporting on these vaccines ranges from 169 (RCV1) to 194 (DTP3). The aggregate percentage coverage is calculated by averaging all reported vaccination rates for each country-year. We did this for all years from 2000 to 2018 (the most recent year for which data are available) to produce a time series of mean vaccination rate.
Table 1.
Ten vaccine doses most commonly reported by countries to WHO
| Rank | No of countries | Vaccine dose | Description |
| 1 | 194 | DTP3 | Third dose of a diphtheria-tetanus-pertussis containing vaccine |
| 2 | 194 | MCV1 | First dose of a measles containing vaccine |
| 3 | 194 | Pol3 | Third dose of a polio vaccine |
| 4 | 192 | Hib3 | Third dose of an Haemophilus influenzae type b vaccine |
| 5 | 188 | DTP1 | First dose of a diphtheria-tetanus-pertussis containing vaccine |
| 6 | 188 | HepB3 | Third dose of a hepatitis B vaccine |
| 7 | 174 | IPV1 | First dose of the inactivated polio vaccine |
| 8 | 171 | BCG | Bacille Calmette-Guérin |
| 9 | 170 | MCV2 | Second dose of a measles containing vaccine |
| 10 | 169 | RCV1 | First dose of a rubella containing vaccine |
In addition, the WHO reports country-years in which the reported coverage rate was greater than 100% with an asterisk to indicate possible false reporting. Those country-years we set to null and did not use in our calculation of country-year averages.
Control variables and covariates
Logged per capita GDP is included to account for differences in the economic capacity of different countries to provide comprehensive vaccinations (ie, a rough proxy for the access and affordability dimensions of vaccine acceptance).33 We use the expected years of schooling measure of education from Unesco’s Human Development Reports in order to measure the general level of knowledge in the population as a rough proxy for the awareness dimension of vaccine acceptance.34
In addition, we include dummy variables representing 10 world regions in order to accommodate fundamental structural differences across regions with regard to health outcomes.35 Finally, we control for level of internet penetration in the country in order to condition the effects of social media variables on how prevalent internet usage is in the first place. For this measure, we use the estimates of per cent internet penetration from the World Development Indicators.33 36
Regressions
We test the relationship between our selected dependent and independent variables using three basic sets of regression frameworks. In this section, we detail the specifications of each regression.
Time series testing the effect of foreign disinformation on mean vaccination rates
First, in order to examine the relationship between the outcome of mean vaccination rates and disinformation campaigns, we performed a cross-country time series analysis. In order to compensate for the serial correlation (ie, the best predictor of 1 year’s value is the previous year's), we include the mean vaccination rate lagged 1 year as an independent variable as well. We run two specifications of this model, one without country fixed effects (Model 1 in table 2) and one with (Model 2 in table 2). The latter is more robust as the array of country-dummy variables controls for any country-specific factors that we have not otherwise controlled for.
Table 2.
Regression results of specified models
| Dependent variable | ||||
| Mean vaccination rate (%) | Belief that vaccines are unsafe (%) | Negative vaccine tweets | ||
| OLS | OLS | Negativebinomial | ||
| (Model 1) | (Model 2) | (Model 3) | (Model 4) | |
| Foreign disinformation | −0.793*** (0.215) | −1.932*** (0.578) | −0.091 (0.831) | 0.293*** (0.112) |
| Social media organisation of offline action | 1.437** (0.604) | |||
| % Internet penetration | 0.001 (0.008) | −0.011 (0.010) | −0.076* (0.043) | 0.004 (0.006) |
| Logged per capita GDP | −0.004 (0.123) | 0.611*** (0.187) | 1.333* (0.711) | −0.074 (0.082) |
| Expected years of schooling | 0.299*** (0.073) | 0.860*** (0.149) | 0.236 (0.325) | −0.020 (0.043) |
| Lagged mean vaccination rate | 0.729*** (0.012) | 0.514*** (0.015) | ||
| 2008 mean vaccination rate | −0.092 (0.060) | 0.007 (0.008) | ||
| Constant | 22.455*** (1.485) | 37.048*** (2.795) | 3.119 (6.747) | −8.292*** (0.862) |
| Region FEs | Yes | No | Yes | Yes |
| Country FEs | No | Yes | No | No |
| Observations | 2902 | 2902 | 137 | 166 |
| Adjusted R2 | 0.707 | 0.733 | 0.199 | |
| Log likelihood | −988.260 | |||
*p<0.1; **p<0.05;***p<0.01.
FEs, fixed effects; OLS, ordinary least squares.
Linear regression testing the effect of social media usage on perception of vaccine safety
Second, in order to examine the relationship between organisation on social media and public perceptions of vaccine safety, we performed an ordinary least squares (OLS) regression. Because the Wellcome polling regarding vaccine safety is only available for 2018, this analysis cannot be done in time series format. We therefore used the mean vaccination rate from 2008 as the control variable for the level of vaccination 'pre-treatment' (in the sense of social media being the treatment variable). These results are presented in Model 3 of table 2.
Count regression testing the effect of foreign disinformation on negative content about vaccines on social media
Third, in order to examine whether disinformation drives negative social media content about vaccinations, we used a negative binomial count regression with the number of negative vaccination tweets as the dependent variable. In addition to the previously noted covariates, this specification uses as an offset variable the total number of tweets from the country in question This is shown in Model 4 in table 2.
Results
In a finding that is robust across all specifications, the prevalence of foreign disinformation operations is highly statistically and substantively significant in predicting a drop in mean vaccination rates over time. In Model 2, a 1-point shift upwards in the 5-point disinformation scale is associated with nearly a 2-percentage point drop in the mean vaccination rate year over year. If we run this same analysis outside a time series context using a 10-year lag as a control variable, we see consistent results with an effect of about 12 percentage points per decade drop in mean vaccination rates, all else being equal. Figure 1 shows the model-predicted level of mean vaccination rate for observed country-year values of foreign disinformation (individual observations marked by ticks on the horizontal axis), with 95% confidence intervals.
Figure 1.
Model estimated effect of foreign disinformation on mean vaccination rate.
Model 3 shows the relationship between social media organisation and the belief that vaccinations are unsafe. Note that while foreign disinformation is not significant at all in this regression, the use of social media to organise offline action is highly predictive of this belief. As such, we cannot simply blame foreign disinformation on the rise of vaccine hesitancy, but must confront the fact that belief in vaccine safety is being driven by organisation on social media.
Figure 2 renders this starkly, showing the distribution of beliefs in vaccines being unsafe against the level of organisation that occurs on social media, with such beliefs mounting as more organisation occurs on social media. If we instead use as a dependent variable the belief that vaccines are unsafe among the disaggregated subpopulations of the Wellcome survey, we can tease out additional important details. First, the regressions remain robust for every other subpopulation except for individuals who identify as secular (as opposed to religious), in which case there is no relationship whatsoever between organisation on social media and disbelief in vaccine safety. Second, while the relationship holds in all other populations, the magnitude of the coefficient shows patterns in line with theoretical expectations: decreasing with age category, income quintile and education level.
Figure 2.
Relationship between disbelief in vaccine safety and use of social media to organise offline action.
Finally, in Model 4 we find support for the connection of foreign disinformation with negative social media activity about vaccination. The substantive effect of foreign disinformation is to increase the number of negative vaccine tweets by 15% for the median country.
Discussion
Statement of principal findings
Our large-n cross-national analysis shows a significant relationship between organisation on social media and public doubts of vaccine safety. It also shows a substantial relationship between foreign disinformation campaigns and declining vaccination rates.
Strengths and weaknesses of the study
Most previous publications report single country data analyses or analyses of data from a few countries combined, but this study is unique in providing a global cross-national analysis of the effect of social media on vaccine hesitancy. Despite the multiple number of vaccine antigens and doses in national vaccination schedules, most studies use the uptake of specific vaccine doses such as DTP3 or MCV1 as a measure of a country’s vaccination coverage.37 38 We use an innovative measure of vaccine uptake that combines coverage with the 10 most commonly reported vaccine doses from countries.
That said, we readily admit that the strengths of this research project are intertwined with some weaknesses. Globally available cross-national data in a project like this are almost invariably blunter instruments than we would prefer. Our measure of foreign disinformation campaigns, while compelling, is simply a general measure of any sort of foreign disinformation campaigns, while anti-vaccination disinformation is a specific and non-random subset of that disinformation. For instance, anti-vaccination content is a distinct form of disinformation pushed by Russian pseudo-state agents for specific strategic goals, but there is no evidence that other leading purveyors of disinformation (such as China or Iran) are doing the same. An index that captured specifically the prevalence of anti-vaccination propaganda would be a more precise test.
Our analysis of Twitter data has similar shortcomings: in order to support the many languages necessary for meaningful analysis, simple sentiment analysis tools of word count-based polarity must be used instead of far more nuanced tools of natural language processing that can capture subtleties of opinion. In addition, Twitter is not used in all countries, so the results found here will be skewed towards findings relevant to those countries. The use of Facebook data, possibly the link sharing data being made available via the Social Science One initiative, might be a feasible extension of this work.
Finally, the available polling data on public attitudes towards the safety of vaccines are only available as a point in time so time series analysis is not possible. In particular, the polling is contemporary enough that it occurs after the potential treatment effect of the last decade of social media. While we use the mean vaccine rate from 2008 as a proxy for attitudes, better and earlier polling data would be ideal.
Meaning of the study
Foreign disinformation campaigns are robustly associated with declines in mean vaccination rates. The use of social media to organise offline action is highly associated with an increase in public belief in vaccines being unsafe. Both of these findings suggest that combating disinformation and misinformation regarding vaccines online is critical to reversing the growth in vaccine hesitancy around the world.
These findings are especially salient in the context of the COVID-19 pandemic, given that the vaccines under development will require deployment globally to billions of people in the next year. Policymakers need to begin planning now for ways to work against the patterns found in this study. While public outreach and education about the importance of vaccines will likely be the cornerstone of any COVID-19 vaccine deployment, we argue that the findings in this paper show that such efforts are empirically not sufficient even if clearly necessary. Based on our findings, we argue for an additional two-pronged strategy explicitly targeting foreign disinformation campaigns and the use of social media by anti-vaccination groups.
First, governments must mandate that social media companies are responsible for taking down anti-vaccination content (whether originating from genuine domestic actors or foreign propaganda operations). This is obviously easier said than done and involves both legal and technical hurdles. However, authoritarian states provide an ironic roadmap despite the chilling implications for free speech: they have consistently been successful at pressing technology companies into policing speech on their behalf within their borders. Where there is political will, there is the capacity for removing content damaging public health.
Second, foreign disinformation campaigns should be addressed at their source. A preponderance of such campaigns amplifying anti-vaccination content originate from within Russia or via pseudo-state actors informally associated with Russia. The utilisation of information warfare on the internet is a broad issue that has little hope of general resolution. However, given the global nature of the COVID-19 crisis and the fact that pushing anti-vaccination propaganda will tangibly cause civilian deaths around the world—and in Russia—there is a chance that, with sufficient pressure and incentivisation, diplomacy could produce a ceasefire of sorts with regard to this specific genre of disinformation.
We urge policymakers to take the time before a COVID-19 vaccine is available for mass distribution as an opportunity for action against social media factors contributing to vaccine hesitancy.
Acknowledgments
The authors thank the Council on Foreign Relations and Global Health Centre for organizing this collaboration and offering input, Kyle Marquardt for invaluable feedback, and five anonymous reviewers who helped strengthen the final version of the article.
Footnotes
Handling editor: Seye Abimbola
Twitter: @slloydwilson
Correction notice: This article has been corrected since it published online to fix table 2 layout.
Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests: None declared.
Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication: Not required.
Provenance and peer review: Commissioned; externally peer reviewed.
Data availability statement: All data except the tweets are available in a public, open access repository. Complete list of unique ids of vaccination related tweets are available for replication purposes upon request.
References
- 1.Dubé E, Laberge C, Guay M, et al. . Vaccine hesitancy: an overview. Hum Vaccin Immunother 2013;9:1763–73. 10.4161/hv.24657 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Dubé E, Gagnon D, Nickels E, et al. . Mapping vaccine hesitancy--country-specific characteristics of a global phenomenon. Vaccine 2014;32:6649–54. 10.1016/j.vaccine.2014.09.039 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.MacDonald NE, Eskola J, Liang X, SAGE Working Group on Vaccine Hesitancy . Vaccine hesitancy: definition, scope and determinants. Vaccine 2015;33:4161–4. 10.1016/j.vaccine.2015.04.036 [DOI] [PubMed] [Google Scholar]
- 4.Mesch GS, Schwirian KP. Social and political determinants of vaccine hesitancy: lessons learned from the H1N1 pandemic of 2009-2010. Am J Infect Control 2015;43:1161–5. 10.1016/j.ajic.2015.06.031 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Larson HJ, de Figueiredo A, Xiahong Z, et al. . The state of vaccine confidence 2016: global insights through a 67-country survey. EBioMedicine 2016;12:295–301. 10.1016/j.ebiom.2016.08.042 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Marti M, de Cola M, MacDonald NE, et al. . Assessments of global drivers of vaccine hesitancy in 2014 –looking beyond safety concerns. PLoS One 2017;12:e0172310–2. 10.1371/journal.pone.0172310 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Lane S, MacDonald NE, Marti M, et al. . Vaccine hesitancy around the globe: analysis of three years of WHO/UNICEF Joint Reporting Form data-2015-2017. Vaccine 2018;36:3861–7. 10.1016/j.vaccine.2018.03.063 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Dubé E, Gagnon D, MacDonald N, et al. . Underlying factors impacting vaccine hesitancy in high income countries: a review of qualitative studies. Expert Rev Vaccines 2018;17:989–1004. 10.1080/14760584.2018.1541406 [DOI] [PubMed] [Google Scholar]
- 9.Davies P, Chapman S, Leask J. Antivaccination activists on the world wide web. Arch Dis Child 2002;87:22–5. 10.1136/adc.87.1.22 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Keelan J, Pavri-Garcia V, Tomlinson G, et al. . YouTube as a source of information on immunization: a content analysis. JAMA 2007;298:2481–4. 10.1001/jama.298.21.2482 [DOI] [PubMed] [Google Scholar]
- 11.Ache KA, Wallace LS. Human papillomavirus vaccination coverage on YouTube. Am J Prev Med 2008;35:389–92. 10.1016/j.amepre.2008.06.029 [DOI] [PubMed] [Google Scholar]
- 12.Keelan J, Pavri V, Balakrishnan R, et al. . An analysis of the human papilloma virus vaccine debate on MySpace blogs. Vaccine 2010;28:1535–40. 10.1016/j.vaccine.2009.11.060 [DOI] [PubMed] [Google Scholar]
- 13.Tafuri S, Gallone MS, Cappelli MG, et al. . Addressing the anti-vaccination movement and the role of HCWs. Vaccine 2014;32:4860–5. 10.1016/j.vaccine.2013.11.006 [DOI] [PubMed] [Google Scholar]
- 14.Mitra T, Counts S, Pennebaker JW. Understanding anti-vaccination attitudes in social media. Tenth International AAAI Conference on Web and Social Media, 2016. [Google Scholar]
- 15.Evrony A, Caplan A. The overlooked dangers of anti-vaccination groups’ social media presence. Hum Vaccin Immunother 2017;13:1475–6. 10.1080/21645515.2017.1283467 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Smith N, Graham T. Mapping the anti-vaccination movement on Facebook. Inf Commun Soc 2019;22:1310–27. 10.1080/1369118X.2017.1418406 [DOI] [Google Scholar]
- 17.Hoffman BL, Felter EM, Chu K-H, et al. . It's not all about autism: the emerging landscape of anti-vaccination sentiment on Facebook. Vaccine 2019;37:2216–23. 10.1016/j.vaccine.2019.03.003 [DOI] [PubMed] [Google Scholar]
- 18.Horowitz DL. Ethnic groups in conflict, 1985. [Google Scholar]
- 19.Broniatowski DA, Jamison AM, Qi S, et al. . Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate. Am J Public Health 2018;108:1378–84. 10.2105/AJPH.2018.304567 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kirk K. How Russia sows confusion in the US vaccine debate, 2019. Foreign Policy [Internet]. Available: https://foreignpolicy.com/2019/04/09/in-the-united-states-russian-trolls-are-peddling-measles-disinformation-on-twitter/
- 21.Fidler D. Disinformation and disease: social media and the ebola epidemic in the Democratic Republic of the Congo, 2019. Council on Foreign Relations [Internet]. Available: https://www.cfr.org/blog/disinformation-and-disease-social-media-and-ebola-epidemic-democratic-republic-congo
- 22.MacKinnon A. Russian disinformation takes on coronavirus, pointing a finger at the United States, 2020. Foreign policy. Available: https://foreignpolicy.com/2020/02/14/russia-blame-america-coronavirus-conspiracy-theories-disinformation/
- 23.Thomson A, Robinson K, Vallée-Tourangeau G. The 5As: a practical taxonomy for the determinants of vaccine uptake. Vaccine 2016;34:1018–24. 10.1016/j.vaccine.2015.11.065 [DOI] [PubMed] [Google Scholar]
- 24.Wilson SL. Geocoded twitter archive for the social sciences, 2020. [Google Scholar]
- 25.Wilson SL. Social media as social science data (forthcoming), 2020. [Google Scholar]
- 26.Al-Rfou R Polyglot, 2020. Available: http://polyglot.readthedocs.org/
- 27.Chen Y, Skiena S. Building sentiment lexicons for all major languages. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Short Papers), 2014:383–9. [Google Scholar]
- 28.Mechkova V, Pemstein D, Seim B, et al. . Digital Society Project dataset V2, 2020. Available: http://digitalsocietyproject.org/data/
- 29.Coppedge M, Gerring J, Knutsen CH, et al. . V-Dem [country–year/country–date] dataset v10. Varieties of Democracy Project 2020.
- 30.Pemstein D, Marquardt KL, Tzelgov E. The V-Dem measurement model: latent variable analysis for cross-national and cross-temporal expert-coded data. V-Dem Working Paper Series 2020.
- 31.Wellcome Wellcome Global Monitor, 2018. Available: https://wellcome.ac.uk/reports/wellcome-global-monitor/2018
- 32.WHO Reported estimates of vaccine coverage, 2019. Available: http://www.who.int/entity/immunization/monitoring_surveillance/data/coverage_series.xls
- 33.Teorell J, Dahlberg S, Holmberg S. The quality of government standard dataset. The Quality of Government Institute, 2020. [Google Scholar]
- 34.UNESCO Human development data (1990-2018); human development reports. Available: http://hdr.undp.org/en/data
- 35.Hadenius A, Teorell J. Pathways from authoritarianism. J Democr 2007;18:143–57. 10.1353/jod.2007.0009 [DOI] [Google Scholar]
- 36.WDI World Development Indicators, 2020. [Google Scholar]
- 37.Mosser JF, Gagne-Maynard W, Rao PC, et al. . Mapping diphtheria-pertussis-tetanus vaccine coverage in Africa, 2000-2016: a spatial and temporal modelling study. Lancet 2019;393:1843–55. 10.1016/S0140-6736(19)30226-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Oyo-Ita A, Wiysonge CS, Oringanje C, et al. . Interventions for improving coverage of childhood immunisation in low- and middle-income countries. Cochrane Database Syst Rev 2016;7:CD008145. 10.1002/14651858.CD008145.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]