Evaluating Prebunking and Nudge Techniques in Tackling Misinformation: A Between-Subject Study on Social Media Platforms
Authors: 
Dipto Barman, Owen ConlanAuthors Info & Claims
Pages 167 - 177
Abstract
Combating misinformation on social media is critical, with preemptive strategies like prebunking and nudging gaining prominence. This paper evaluates the effectiveness of nudge and prebunking strategies in enhancing individuals' ability to distinguish between misinformation and factual content and their confidence in their accuracy judgments. Employing a between-subject experimental design, participants (N = 328) were categorised into three conditions: a control condition with no intervention, a nudge condition exposed to behavioural cues to promote critical scrutiny of information and a prebunking condition receiving implicit context about the claim. The results indicate that the prebunking message improves the identification of misinformation and confidence in judgments compared to nudging and control interventions. No significant difference was observed between the nudge and control groups regarding judgment accuracy or confidence. However, individual differences in interventions were noted. The study reveals that deliberate thinkers require some form of intervention to discern false news claims effectively. Moreover, it was found that participants with right-leaning political views were less influenced by prebunking messages, suggesting that nudges might be more effective for this demographic. These findings highlight the necessity of adopting a user-centric approach that considers individual characteristics to tailor interventions that may be required to combat misinformation effectively among diverse user groups on social media platforms.
References
[1]
Sacha Altay, Anne-Sophie Hacquin, and Hugo Mercier. 2022. Why do so few people share fake news? It hurts their reputation. New Media & Society 24, 6 (June 2022), 1303–1324. https://doi.org/10.1177/1461444820969893
[2]
Evangelia Anagnostopoulou, Babis Magoutas, Efthimios Bothos, Johann Schrammel, Rita Orji, and Gregoris Mentzas. 2017. Exploring the Links Between Persuasion, Personality and Mobility Types in Personalized Mobility Applications. In Persuasive Technology: Development and Implementation of Personalized Technologies to Change Attitudes and Behaviors, Peter W. de Vries, Harri Oinas-Kukkonen, Liseth Siemons, Nienke Beerlage-de Jong and Lisette van Gemert-Pijnen (eds.). Springer International Publishing, Cham, 107–118. https://doi.org/10.1007/978-3-319-55134-0_9
[3]
Simge Andı and Jesper Akesson. 2021. Nudging Away False News: Evidence from a Social Norms Experiment. Digital Journalism 9, 1 (January 2021), 106–125. https://doi.org/10.1080/21670811.2020.1847674
[4]
Sumitra Badrinathan and Simon Chauchard. 2023. “I Don't Think That's True, Bro!” Social Corrections of Misinformation in India. The International Journal of Press/Politics (February 2023), 194016122311587. https://doi.org/10.1177/19401612231158770
[5]
Meital Balmas. 2014. When Fake News Becomes Real: Combined Exposure to Multiple News Sources and Political Attitudes of Inefficacy, Alienation, and Cynicism. Communication Research 41, 3 (April 2014), 430–454. https://doi.org/10.1177/0093650212453600
[6]
Dipto Barman and Owen Conlan. 2023. Does Explanation Matter? An Exploratory Study on the Effects of Covid–19 Misinformation Warning Flags on Social Media. In 2023 10th International Conference on Behavioural and Social Computing (BESC), October 30, 2023. IEEE, Larnaca, Cyprus, 1–7. https://doi.org/10.1109/BESC59560.2023.10386371
[7]
Melisa Basol, Jon Roozenbeek, Manon Berriche, Fatih Uenal, William P. McClanahan, and Sander van der Linden. 2021. Towards psychological herd immunity: Cross-cultural evidence for two prebunking interventions against COVID-19 misinformation. Big Data & Society 8, 1 (January 2021), 205395172110138. https://doi.org/10.1177/20539517211013868
[8]
Daniele Bellutta, Joshua Uyheng, and Kathleen M. Carley. 2022. The Missing Link Between User Engagement and Misinformation's Impact on Online Behavior. In Social, Cultural, and Behavioral Modeling, Robert Thomson, Christopher Dancy and Aryn Pyke (eds.). Springer International Publishing, Cham, 79–89. https://doi.org/10.1007/978-3-031-17114-7_8
[9]
Mauro Bertolotti and Patrizia Catellani. 2022. Counterfactual thinking as a prebunking strategy to contrast misinformation on COVID-19. Journal of Experimental Social Psychology (September 2022), 104404. https://doi.org/10.1016/j.jesp.2022.104404
[10]
A. Bessi. 2016. Personality traits and echo chambers on facebook. Comput. Hum. Behav. 65, (2016), 319–324. https://doi.org/10.1016/j.chb.2016.08.016
[11]
Mikey Biddlestone, Jon Roozenbeek, and Sander van der Linden. 2022. Once (but not twice) upon a time: Narrative inoculation against conjunction errors indirectly reduces conspiracy beliefs and improves truth discernment. Applied Cognitive Psychology (December 2022), acp.4025. https://doi.org/10.1002/acp.4025
[12]
Martin Bruder, Peter Haffke, Nick Neave, Nina Nouripanah, and Roland Imhoff. 2013. Measuring Individual Differences in Generic Beliefs in Conspiracy Theories Across Cultures: Conspiracy Mentality Questionnaire. Front. Psychol. 4, (2013). https://doi.org/10.3389/fpsyg.2013.00225
[13]
T. Buchanan and J. Kempley. 2021. Individual differences in sharing false political information on social media: Direct and indirect effects of cognitive-perceptual schizotypy and psychopathy. Pers. Individ. Differ. 182, (2021). https://doi.org/10.1016/j.paid.2021.111071
[14]
Ruth M. J. Byrne. 2019. Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, August 2019. International Joint Conferences on Artificial Intelligence Organization, Macao, China, 6276–6282. https://doi.org/10.24963/ijcai.2019/876
[15]
Josh Compton. 2020. Prophylactic Versus Therapeutic Inoculation Treatments for Resistance to Influence. Communication Theory 30, 3 (August 2020), 330–343. https://doi.org/10.1093/ct/qtz004
[16]
John Cook, Stephan Lewandowsky, and Ullrich K. H. Ecker. 2017. Neutralizing misinformation through inoculation: Exposing misleading argumentation techniques reduces their influence. PLoS ONE 12, 5 (May 2017), e0175799. https://doi.org/10.1371/journal.pone.0175799
[17]
Shih-Chieh Dai, Yi-Li Hsu, Aiping Xiong, and Lun-Wei Ku. 2022. Ask to Know More: Generating Counterfactual Explanations for Fake Claims. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, August 14, 2022. ACM, Washington DC USA, 2800–2810. https://doi.org/10.1145/3534678.3539205
[18]
Ullrich K. H. Ecker, Ziggy O'Reilly, Jesse S. Reid, and Ee Pin Chang. 2020. The effectiveness of short-format refutational fact-checks. British Journal of Psychology 111, 1 (2020), 36–54. https://doi.org/10.1111/bjop.12383
[19]
Álex Escolà-Gascón, Neil Dagnall, Andrew Denovan, Kenneth Drinkwater, and Miriam Diez-Bosch. 2023. Who falls for fake news? Psychological and clinical profiling evidence of fake news consumers. Personality and Individual Differences 200, (January 2023), 111893. https://doi.org/10.1016/j.paid.2022.111893
[20]
Franz Faul, Edgar Erdfelder, Albert-Georg Lang, and Axel Buchner. 2007. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods 39, 2 (May 2007), 175–191. https://doi.org/10.3758/BF03193146
[21]
Lisa Fazio. 2020. Pausing to consider why a headline is true or false can help reduce the sharing of false news. HKS Misinfo Review (February 2020). https://doi.org/10.37016/mr-2020-009
[22]
Seth Flaxman, Sharad Goel, and Justin M. Rao. 2016. Filter Bubbles, Echo Chambers, and Online News Consumption. Public Opinion Quarterly 80, S1 (January 2016), 298–320. https://doi.org/10.1093/poq/nfw006
[23]
Shane Frederick. 2005. Cognitive Reflection and Decision Making. Journal of Economic Perspectives 19, 4 (November 2005), 25–42. https://doi.org/10.1257/089533005775196732
[24]
R. K. Garrett. 2019. Social media's contribution to political misperceptions in US Presidential elections. PloS ONE 14, 3 (March 2019). https://doi.org/10.1371/journal.pone.0213500
[25]
Andrew M. Guess, Michael Lerner, Benjamin Lyons, Jacob M. Montgomery, Brendan Nyhan, Jason Reifler, and Neelanjan Sircar. 2020. A digital media literacy intervention increases discernment between mainstream and false news in the United States and India. Proc. Natl. Acad. Sci. U.S.A. 117, 27 (July 2020), 15536–15545. https://doi.org/10.1073/pnas.1920498117
[26]
Sajanee Halko and Julie A. Kientz. 2010. Personality and Persuasive Technology: An Exploratory Study on Health-Promoting Mobile Applications. In Persuasive Technology, Thomas Ploug, Per Hasle and Harri Oinas-Kukkonen (eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 150–161. https://doi.org/10.1007/978-3-642-13226-1_16
[27]
Lynn Hasher, David Goldstein, and Thomas Toppino. 1977. Frequency and the conference of referential validity. Journal of Verbal Learning and Verbal Behavior 16, 1 (February 1977), 107–112. https://doi.org/10.1016/S0022-5371(77)80012-1
[28]
Aumyo Hassan and Sarah J. Barber. 2021. The effects of repetition frequency on the illusory truth effect. Cognitive Research: Principles and Implications 6, 1 (May 2021), 38. https://doi.org/10.1186/s41235-021-00301-5
[29]
Yi-Li Hsu, Shih-Chieh Dai, Aiping Xiong, and Lun-Wei Ku. 2023. Is Explanation the Cure? Misinformation Mitigation in the Short Term and Long Term. Retrieved November 20, 2023 from http://arxiv.org/abs/2310.17711
[30]
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of Hallucination in Natural Language Generation. ACM Comput. Surv. 55, 12 (December 2023), 1–38. https://doi.org/10.1145/3571730
[31]
Kiemute Oyibo, Rita Orji, Jaap Ham, Joshua Nwokeji, and Ana Ciocarlan. 2021. Personalizing Persuasive Technologies: Personalization for Wellbeing. (2021). https://doi.org/10.13140/RG.2.2.26605.00485
[32]
Loukas Konstantinou, Dionysis Panos, and Evangelos Karapanos. 2024. Exploring the Design of Technology-Mediated Nudges for Online Misinformation. International Journal of Human–Computer Interaction (January 2024), 1–28. https://doi.org/10.1080/10447318.2023.2301265
[33]
Mehrdad Koohikamali and Anna Sidorova. 2017. Information Re-Sharing on Social Network Sites in the Age of Fake News. InformingSciJ 20, (2017), 215–235. https://doi.org/10.28945/3871
[34]
Anastasia Kozyreva, Philipp Lorenz-Spreen, Stefan Michael Herzog, Ullrich K. H. Ecker, Stephan Lewandowsky, and Ralph Hertwig. 2022. Toolbox of Interventions Against Online Misinformation and Manipulation. PsyArXiv. https://doi.org/10.31234/osf.io/x8ejt
[35]
Candice Lanius, Ryan Weber, and William I. MacKenzie. 2021. Use of bot and content flags to limit the spread of misinformation among social networks: a behavior and attitude survey. Soc. Netw. Anal. Min. 11, 1 (December 2021), 32. https://doi.org/10.1007/s13278-021-00739-x
[36]
David M J Lazer, Matthew A Baum, Yochai Benkler, Adam J Berinsky, Kelly M Greenhill, Filippo Menczer, Miriam J Metzger, Brendan Nyhan, Gordon Pennycook, David Rothschild, Michael Schudson, Steven A Sloman, Cass R Sunstein, Emily A Thorson, Duncan J Watts, and Jonathan L Zittrain. 2018. The science of fake news. Science 359, 6380 (March 2018), 1094. https://doi.org/10.1126/science.aao2998
[37]
Stephan Lewandowsky, John Cook, and Doug Lombardi. 2020. Debunking Handbook 2020. https://doi.org/10.17910/B7.1182
[38]
Stephan Lewandowsky and Sander van der Linden. 2021. Countering Misinformation and Fake News Through Inoculation and Prebunking. European Review of Social Psychology (February 2021), 1–38. https://doi.org/10.1080/10463283.2021.1876983
[39]
Han Li. 2022. Disinformation Targeting Chinese Americans Sparks Concerns. The San Francisco Standard. Retrieved October 24, 2023 from https://sfstandard.com/2022/10/07/disinformation-chinese-americans-social-media-elections/
[40]
Rakoen Maertens, Friedrich M. Götz, Hudson F. Golino, Jon Roozenbeek, Claudia R. Schneider, Yara Kyrychenko, John R. Kerr, Stefan Stieger, William P. McClanahan, Karly Drabot, James He, and Sander Van Der Linden. 2023. The Misinformation Susceptibility Test (MIST): A psychometrically validated measure of news veracity discernment. Behav Res (June 2023). https://doi.org/10.3758/s13428-023-02124-2
[41]
C. Martel, G. Pennycook, and D. G. Rand. 2020. Reliance on emotion promotes belief in fake news. Cogn. Res: Prin. Implic. 5, 1 (2020). https://doi.org/10.1186/s41235-020-00252-3
[42]
W. J. McGuire and D. Papageorgis. 1961. The relative efficacy of various types of prior belief-defense in producing immunity against persuasion. The Journal of Abnormal and Social Psychology 62, 2 (March 1961), 327–337. https://doi.org/10.1037/h0042026
[43]
Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence 267, (February 2019), 1–38. https://doi.org/10.1016/j.artint.2018.07.007
[44]
Gillian Murphy, Elizabeth F. Loftus, Rebecca Hofstein Grady, Linda J. Levine, and Ciara M. Greene. 2019. False Memories for Fake News During Ireland's Abortion Referendum. Psychol Sci 30, 10 (October 2019), 1449–1459. https://doi.org/10.1177/0956797619864887
[45]
Jessica Murray, Aina J. Khan, and Rajeev Syal. 2022. ‘It feels like people want to fight’: how communal unrest flared in Leicester. The Guardian. Retrieved October 24, 2023 from https://www.theguardian.com/uk-news/2022/sep/23/how-communal-unrest-flared-leicester-muslim-hindu-tensions
[46]
Brendan Nyhan and Jason Reifler. 2010. When Corrections Fail: The Persistence of Political Misperceptions. Polit Behav 32, 2 (June 2010), 303–330. https://doi.org/10.1007/s11109-010-9112-2
[47]
Katherine Ognyanova, David Lazer, Ronald E. Robertson, and Christo Wilson. 2020. Misinformation in action: Fake news exposureis linked to lower trust in media, higher trust in government when your side is in power. HKS Misinfo Review (June 2020). https://doi.org/10.37016/mr-2020-024
[48]
OpenAI et. al. 2023. GPT-4 Technical Report. (2023). https://doi.org/10.48550/ARXIV.2303.08774
[49]
Gordon Pennycook, Jabin Binnendyk, Christie Newton, and David G. Rand. 2021. A Practical Guide to Doing Behavioral Research on Fake News and Misinformation. Collabra: Psychology 7, 1 (July 2021), 25293. https://doi.org/10.1525/collabra.25293
[50]
Gordon Pennycook, Jonathon McPhetres, Yunhao Zhang, Jackson G Lu, and David G Rand. Fighting COVID-19 Misinformation on Social Media: Experimental Evidence for a Scalable Accuracy-Nudge Intervention. 11.
[51]
Gordon Pennycook and David G. Rand. 2020. Who falls for fake news? The roles of bullshit receptivity, overclaiming, familiarity, and analytic thinking. J Pers 88, 2 (April 2020), 185–200. https://doi.org/10.1111/jopy.12476
[52]
Megan A.K. Peters. 2022. Confidence in Decision-Making. In Oxford Research Encyclopedia of Neuroscience. Oxford University Press. https://doi.org/10.1093/acrefore/9780190264086.013.371
[53]
20 Oct 2023 Associated Press. 2023. Misinformation About the Israel-Hamas War Is Flooding Social Media. Here Are the Facts. Military.com. Retrieved October 23, 2023 from https://www.military.com/daily-news/2023/10/20/misinformation-about-israel-hamas-war-flooding-social-media-here-are-facts.html
[54]
Jon Roozenbeek, Alexandra L. J. Freeman, and Sander van der Linden. 2021. How Accurate Are Accuracy-Nudge Interventions? A Preregistered Direct Replication of Pennycook (2020). Psychol Sci 32, 7 (July 2021), 1169–1178. https://doi.org/10.1177/09567976211024535
[55]
Jon Roozenbeek and Sander van der Linden. 2019. Fake news game confers psychological resistance against online misinformation. Palgrave Commun 5, 1 (December 2019), 65. https://doi.org/10.1057/s41599-019-0279-9
[56]
Jon Roozenbeek, Jane Suiter, and Eileen Culloty. 2022. Countering Misinformation: Evidence, Knowledge Gaps, and Implications of Current Interventions. PsyArXiv. https://doi.org/10.31234/osf.io/b52um
[57]
M. Sallam, D. Dababseh, A. Yaseen, A. Al-Haidar, D. Taim, H. Eid, N. A. Ababneh, F. G. Bakri, and A. Mahafzah. 2020. COVID-19 misinformation: Mere harmless delusions or much more? A knowledge and attitude cross-sectional study among the general public residing in Jordan. PLoS ONE 15, 12 December (2020). https://doi.org/10.1371/journal.pone.0243264
[58]
Nikita A. Salovich, Amalia M. Donovan, Scott R. Hinze, and David N. Rapp. 2021. Can confidence help account for and redress the effects of reading inaccurate information? Mem Cogn 49, 2 (February 2021), 293–310. https://doi.org/10.3758/s13421-020-01096-4
[59]
Cornelia Sindermann, Helena Sophia Schmitt, Dmitri Rozgonjuk, Jon D. Elhai, and Christian Montag. 2021. The evaluation of fake and true news: on the role of intelligence, personality, interpersonal trust, ideological attitudes, and news consumption. Heliyon 7, 3 (March 2021), e06503. https://doi.org/10.1016/j.heliyon.2021.e06503
[60]
Christina Steindl, Eva Jonas, Sandra Sittenthaler, Eva Traut-Mattausch, and Jeff Greenberg. 2015. Understanding Psychological Reactance: New Developments and Findings. Zeitschrift für Psychologie 223, 4 (October 2015), 205–214. https://doi.org/10.1027/2151-2604/a000222
[61]
Edson C Tandoc, Richard Ling, Oscar Westlund, Andrew Duffy, Debbie Goh, and Lim Zheng Wei. 2018. Audiences’ acts of authentication in the age of fake news: A conceptual framework. New Media & Society 20, 8 (August 2018), 2745–2763. https://doi.org/10.1177/1461444817731756
[62]
Keela S Thomson and Daniel M Oppenheimer. 2016. Investigating an alternate form of the cognitive reflection test. Judgment and Decision Making 11, 1 (2016), 15.
[63]
Tyler Vergho, Jean-Francois Godbout, Reihaneh Rabbany, and Kellin Pelrine. 2024. Comparing GPT-4 and Open-Source Language Models in Misinformation Mitigation. Retrieved January 22, 2024 from http://arxiv.org/abs/2401.06920
[64]
Nathan Walter and Riva Tukachinsky. 2020. A Meta-Analytic Examination of the Continued Influence of Misinformation in the Face of Correction: How Powerful Is It, Why Does It Happen, and How to Stop It? Communication Research 47, 2 (March 2020), 155–177. https://doi.org/10.1177/0093650219854600
[65]
Sam Wineburg, Joel Breakstone, Sarah McGrew, Mark D. Smith, and Teresa Ortega. 2022. Lateral reading on the open Internet: A district-wide field study in high school government classes. Journal of Educational Psychology 114, 5 (July 2022), 893–909. https://doi.org/10.1037/edu0000740
[66]
Thomas Zerback, Florian Töpfl, and Maria Knöpfle. 2021. The disconcerting potential of online disinformation: Persuasive effects of astroturfing comments and three strategies for inoculation against them. New Media & Society 23, 5 (May 2021), 1080–1098. https://doi.org/10.1177/1461444820908530
Index Terms
- Evaluating Prebunking and Nudge Techniques in Tackling Misinformation: A Between-Subject Study on Social Media Platforms
Index terms have been assigned to the content through auto-classification.
Recommendations
The diffusion of misinformation on social media
This study examines dynamic communication processes of political misinformation on social media focusing on three components: the temporal pattern, content mutation, and sources of misinformation. We traced the lifecycle of 17 popular political rumors ...
Are Mutated Misinformation More Contagious? A Case Study of COVID-19 Misinformation on Twitter
WebSci '22: Proceedings of the 14th ACM Web Science Conference 2022The spread of online misinformation has become a major global risk. Understanding how misinformation propagates on social media is vital. While prior studies suggest that the content factors, such as emotion and topic in texts, are closely related to the ...
Encounters with Visual Misinformation and Labels Across Platforms: An Interview and Diary Study to Inform Ecosystem Approaches to Misinformation Interventions
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing SystemsSocial media platforms face rampant misinformation spread through multimedia posts shared in highly-personalized contexts [10, 11]. Foundational qualitative research is necessary to ensure platforms’ misinformation interventions are aligned with users’ ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
September 2024
415 pages
ISBN:9798400705953
DOI:10.1145/3648188
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 10 September 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
HT '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 378 of 1,158 submissions, 33%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 157Total Downloads
- Downloads (Last 12 months)157
- Downloads (Last 6 weeks)94
Reflects downloads up to 19 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in