Detecting Misinformation on Telegram Anti-vaccine Communities
Resumo
Due to the substantial volume of misinformation regarding COVID-19 in Brazil, this paper proposes the application of machine learning methods to identify false or harmful information in anti-vaccine communities on Telegram. To this end, we developed a dataset of 1,500 messages labeled by experts according to three aspects: veracity level, potential for harm, and category. The labeling process achieved an agreement score of 81%. Experiments were conducted using state-of-the-art algorithms such as XGBoost, a BERT-based classifier, and a GPT-based classifier. The models trained on the labeled dataset achieved an F1-Score of 0.83 for detecting falsehood and 0.92 for potential harm, indicating their effectiveness in identifying misinformation in this context.
Palavras-chave:
misinformation, machine learning, telegram, vaccine
Referências
Bertaglia, T. F. C. and Nunes, M. d. G. V. (2016). Exploring word embeddings for unsupervised textual user-generated content normalization. In 2nd WNUT, pages 112–120.
Callum Jones, S. R. and Robards, B. (2024). White Warriors and Weak Women: Identifying Central Discourses of Masculinity in Neo-Nazi Telegram Channels. Studies in Conflict & Terrorism, pages 1–26.
Cavalini, A., Malini, F., Gouveia, F., and Comarela, G. (2023). Politics and Disinformation: Analyzing the Use of Telegram’s Information Disorder Network in Brazil for Political Mobilization. First Monday, 28(5).
Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2018). BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805.
Gaglani, J., Gandhi, Y., Gogate, S., and Halbe, A. (2020). Unsupervised whatsapp fake news detection using semantic search. In 4th ICICCS, pages 285–289.
Guhl, J. and Davey, J. (2020). A Safe Space to Hate: White Supremacist Mobilisation on Telegram. Institute for Strategic Dialogue (ISD). [link].
ICEPi (2022). Metodologia de Coleta e Modelagem das Redes de Desordem Informacional do Telegram. Technical report, Instituto Capixaba de Ensino, Pesquisa e Inovacação em Saúde.
Júnior, M., Melo, P., Kansaon, D., Mafra, V., Sa, K., and Benevenuto, F. (2022). Telegram Monitor: Monitoring Brazilian Political Groups and Channels on Telegram. In 33rd ACM Conference on Hypertext and Social Media, HT ’22.
Kılıç, J., Yıldırım, M. S., Alakuş, O. F., Kiliç, D. K., Y., N. A., and Ebik, B. (2022). Vaccination hesitancy as a cause of covid-related mortality. International Journal of Research in Medical Sciences, 10(9):1833–1838.
Lee, S. K., Sun, J., Jang, S., and Connelly, S. (2022). Misinformation of COVID-19 Vaccines and Vaccine Hesitancy. Scientific Reports, 12(1):13681.
Machado, C., Kira, B., Narayanan, V., Kollanyi, B., and Howard, P. (2019). A study of misinformation in whatsapp groups with a focus on the brazilian presidential elections. WWW ’19, page 1013–1019, New York, NY, USA.
Malini, F., Sodré, F., Cavalini, A., Herkenhoff, G., and Gouveia, F. (2024). Five patterns of vaccine misinformation on telegram. In 16th International Conference on Advances in Social Networks Analysis and Mining, ASONAM, Italy. (to appear).
Robertson, S. and Jones, S. (1976). Relevance weighting of search terms. Journal of the American Society for Information science, 27:129–146.
Rogers, R. (2020). Deplatforming: Following extreme Internet celebrities to Telegram and alternative social media. Media Studies, 35(3):213–229.
Souza, F., Nogueira, R., and Lotufo, R. (2020). BERTimbau: Pretrained BERT Models for Brazilian Portuguese. In Cerri, R. and Prati, R. C., editors, Intelligent Systems, pages 403–417, Cham. Springer International Publishing.
Wardle, C. and Derakhshan, H. (2017). Information disorder: Toward an interdisciplinary framework for research and policy making. Council of Europe.
Weimann, G. (2016). Terrorist Migration to the Dark Web. Perspectives on Terrorism, 10(3):40–44. JSTOR, [link].
WHO (2022). Health topics: Infodemic. World Health Organization. [link]. Accessed on: May 9, 2023.
Callum Jones, S. R. and Robards, B. (2024). White Warriors and Weak Women: Identifying Central Discourses of Masculinity in Neo-Nazi Telegram Channels. Studies in Conflict & Terrorism, pages 1–26.
Cavalini, A., Malini, F., Gouveia, F., and Comarela, G. (2023). Politics and Disinformation: Analyzing the Use of Telegram’s Information Disorder Network in Brazil for Political Mobilization. First Monday, 28(5).
Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2018). BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805.
Gaglani, J., Gandhi, Y., Gogate, S., and Halbe, A. (2020). Unsupervised whatsapp fake news detection using semantic search. In 4th ICICCS, pages 285–289.
Guhl, J. and Davey, J. (2020). A Safe Space to Hate: White Supremacist Mobilisation on Telegram. Institute for Strategic Dialogue (ISD). [link].
ICEPi (2022). Metodologia de Coleta e Modelagem das Redes de Desordem Informacional do Telegram. Technical report, Instituto Capixaba de Ensino, Pesquisa e Inovacação em Saúde.
Júnior, M., Melo, P., Kansaon, D., Mafra, V., Sa, K., and Benevenuto, F. (2022). Telegram Monitor: Monitoring Brazilian Political Groups and Channels on Telegram. In 33rd ACM Conference on Hypertext and Social Media, HT ’22.
Kılıç, J., Yıldırım, M. S., Alakuş, O. F., Kiliç, D. K., Y., N. A., and Ebik, B. (2022). Vaccination hesitancy as a cause of covid-related mortality. International Journal of Research in Medical Sciences, 10(9):1833–1838.
Lee, S. K., Sun, J., Jang, S., and Connelly, S. (2022). Misinformation of COVID-19 Vaccines and Vaccine Hesitancy. Scientific Reports, 12(1):13681.
Machado, C., Kira, B., Narayanan, V., Kollanyi, B., and Howard, P. (2019). A study of misinformation in whatsapp groups with a focus on the brazilian presidential elections. WWW ’19, page 1013–1019, New York, NY, USA.
Malini, F., Sodré, F., Cavalini, A., Herkenhoff, G., and Gouveia, F. (2024). Five patterns of vaccine misinformation on telegram. In 16th International Conference on Advances in Social Networks Analysis and Mining, ASONAM, Italy. (to appear).
Robertson, S. and Jones, S. (1976). Relevance weighting of search terms. Journal of the American Society for Information science, 27:129–146.
Rogers, R. (2020). Deplatforming: Following extreme Internet celebrities to Telegram and alternative social media. Media Studies, 35(3):213–229.
Souza, F., Nogueira, R., and Lotufo, R. (2020). BERTimbau: Pretrained BERT Models for Brazilian Portuguese. In Cerri, R. and Prati, R. C., editors, Intelligent Systems, pages 403–417, Cham. Springer International Publishing.
Wardle, C. and Derakhshan, H. (2017). Information disorder: Toward an interdisciplinary framework for research and policy making. Council of Europe.
Weimann, G. (2016). Terrorist Migration to the Dark Web. Perspectives on Terrorism, 10(3):40–44. JSTOR, [link].
WHO (2022). Health topics: Infodemic. World Health Organization. [link]. Accessed on: May 9, 2023.
Publicado
14/10/2024
Como Citar
CAVALINI, Athus; DONADIA, Thamya; MALINI, Fabio; COMARELA, Giovanni.
Detecting Misinformation on Telegram Anti-vaccine Communities. In: SIMPÓSIO BRASILEIRO DE BANCO DE DADOS (SBBD), 39. , 2024, Florianópolis/SC.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 729-735.
ISSN 2763-8979.
DOI: https://doi.org/10.5753/sbbd.2024.243188.