Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

I-FLASH: Interpretable Fake News Detector Using LIME and SHAP

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

The rise of social media enables people to disseminate information. However, when false but appealing information is presented as news, it becomes a cause for serious concern as it might lead to a harmful influence on communities of innocent believers. To address this issue, we propose I-FLASH, an interpretable fake news detector that not only detects fake news but also explains why it considers some content fake or genuine. Moreover, recent research evaluated their models for fake news detection on domain-specific datasets. Therefore, in this paper, two new tiny datasets, FactCheck and FactCheck2, were culled from the official Twitter accounts/websites of various well-known media outlets, covering a variety of other societal domains such as education, crime, and technology. We also compared the performance of the machine learning model (logistic regression with TF-IDF), deep learning model (bidirectional LSTM with GloVe word embeddings), and the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model on curated datasets along with two other popular datasets, namely, LIAR and COVID-19. The stratified 10-fold cross-validation accuracy of 94.41 ± 0.38% on the COVID-19 dataset, 61.18 ± 0.55% on the LIAR dataset, 87.25 ± 2.45% on FactCheck, and 92.91 ± 2.07% on FactCheck2, attained at 95% confidence interval, establishes the efficacy of models. On cross-dataset validation, we observe that the model trained on a generalized dataset like FactCheck2 can perform well on domain-specific datasets like COVID-19 and LIAR with a validation accuracy of 64.25% and 54.22%, respectively. Finally, using XAI methods—LIME and SHAP revealed important terms while predicting the news class (fake/real).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Data Availability

All data generated or analyzed during this study are included in this article.

Notes

  1. https://github.com/Vanshika-Dua/I-FLASH.

  2. https://www.indiatoday.in/fact-check.

References

  1. “Fake news in India.” https://en.wikipedia.org/wiki/Fake_news_in_India. Accessed from 10 Mar 2022.

  2. Albahar, M. (2021). A hybrid model for fake news detection: Leveraging news content and user comments in fake news. IET Information Security, 15(2), 169–177.

    Article  Google Scholar 

  3. Zhou, X., & Zafarani, R. (2020). A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys (CSUR), 53(5), 1–40.

    Article  Google Scholar 

  4. Zhang, X., & Ghorbani, A. A. (2020). An overview of online fake news: Characterization, detection, and discussion. Information Processing & Management, 57(2), 102025.

    Article  Google Scholar 

  5. Apuke, O. D., & Omar, B. (2021). Fake news and COVID-19: Modelling the predictors of fake news sharing among social media users. Telematics and Informatics, 56, 101475.

    Article  Google Scholar 

  6. “COVID-19 and WhatsApp Fuel Surge of Fake News in India.” https://thewire.in/tech/covid-19-and-whatsapp-fuel-surge-of-fake-news-in-india. Accessed from 10 Mar 2022.

  7. Pathwar, P., & Gill, S. (2022). Tackling COVID-19 infodemic using deep learning. In Proceedings of international conference on computational intelligence and data engineering (pp. 319–335). Springer.

  8. Ayoub, J., Yang, X. J., & Zhou, F. (2021). Combat COVID-19 infodemic using explainable natural language processing models. Information Processing & Management, 58(4), 102569.

    Article  Google Scholar 

  9. Bovet, A., & Makse, H. A. (2019). Influence of fake news in Twitter during the 2016 US presidential election. Nature Communications, 10(1), 1–14.

    Article  Google Scholar 

  10. Zhou, X., Jain, A., Phoha, V. V., & Zafarani, R. (2020). Fake news early detection: A theory-driven model. Digital Threats: Research and Practice, 1(2), 1–25.

    Article  Google Scholar 

  11. Saleh, H., Alharbi, A., & Alsamhi, S. H. (2021). OPCNN-FAKE: Optimized convolutional neural network for fake news detection. IEEE Access, 9, 129471–129489.

    Article  Google Scholar 

  12. Shahid, W., Li, Y., Staples, D., Amin, G., Hakak, S., & Ghorbani, A. (2022). Are you a cyborg, bot or human? A survey on detecting fake news spreaders. IEEE Access, 10, 27069–27083.

    Article  Google Scholar 

  13. Shao, C., Ciampaglia, G. L., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F. (2018). The spread of low-credibility content by social bots. Nature Communications, 9(1), 1–9.

    Article  Google Scholar 

  14. Kaliyar, R. K., Goswami, A., & Narang, P. (2021). FakeBERT: Fake news detection in social media with a BERT-based deep learning approach. Multimedia Tools and Applications, 80(8), 11765–11788.

    Article  Google Scholar 

  15. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016).“" Why should i trust you?" Explaining the predictions of any classifier”. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1135–1144).

  16. Zhang, C., Gupta, A., Kauten, C., Deokar, A. V., & Qin, X. (2019). Detecting fake news for reducing misinformation risks using analytics approaches. European Journal of Operational Research, 279(3), 1036–1052. https://doi.org/10.1016/j.ejor.2019.06.022

    Article  Google Scholar 

  17. Conroy, N. K., Rubin, V. L., & Chen, Y. (2015). Automatic deception detection: Methods for finding fake news. In Proceedings of the association for information science and technology (vol. 52, pp. 1–4).

  18. Shahzad, K., Khan, S. A., Ahmad, S., & Iqbal, A. (2022). A Scoping review of the relationship of big data analytics with context-based fake news detection on digital media in data age. Sustainability, 14(21), 14365. https://doi.org/10.3390/su142114365

    Article  Google Scholar 

  19. Davoudi, M., Moosavi, M. R., & Sadreddini, M. H. (2022). DSS: A hybrid deep model for fake news detection using propagation tree and stance network. Expert Systems with Applications, 198, 116635.

    Article  Google Scholar 

  20. Zhou, X., & Zafarani, R. (2019). Network-based fake news detection: A pattern-driven approach. ACM SIGKDD Explorations Newsletter, 21(2), 48–60.

    Article  Google Scholar 

  21. Raza, S., & Ding, C. (2022). Fake news detection based on news content and social contexts: A transformer-based approach. International Journal of Data Science and Analytics, 13(4), 335–362.

    Article  Google Scholar 

  22. Szczepański, M., Pawlicki, M., Kozik, R., & Choraś, M. (2021). New explainability method for BERT-based model in fake news detection. Scientific Reports, 11(1), 1–13.

    Article  Google Scholar 

  23. Bhutani, B., Rastogi, N., Sehgal, P., & Purwar, A. (2019). “Fake news detection using sentiment analysis,” In 2019 twelfth international conference on contemporary computing (IC3) (pp. 1–5). IEEE.

  24. Aldwairi, M., & Alwahedi, A. (2018). Detecting fake news in social media networks. Procedia Computer Science, 141, 215–222.

    Article  Google Scholar 

  25. Trueman, T. E., Kumar, A., Narayanasamy, P., & Vidya, J. (2021). Attention-based C-BiLSTM for fake news detection. Applied Soft Computing, 110, 107600.

    Article  Google Scholar 

  26. Nasir, J. A., Khan, O. S., & Varlamis, I. (2021). Fake news detection: A hybrid CNN-RNN based deep learning approach. International Journal of Information Management Data Insights, 1(1), 100007.

    Article  Google Scholar 

  27. Sastrawan, I. K., Bayupati, I., & Arsa, D. M. S. (2021). “Detection of fake news using deep learning CNN–RNN based methods.” ICT Express.

  28. Khan, J. Y., Khondaker, M. T. I., Afroz, S., Uddin, G., & Iqbal, A. (2021). A benchmark study of machine learning models for online fake news detection. Machine Learning with Applications, 4, 100032.

    Article  Google Scholar 

  29. Flores, L. J. Y., & Hao, Y. (2022). “An Adversarial Benchmark for Fake News Detection Models,” arXiv preprintarXiv:2201.00912

  30. Xu, F., Sheng, V. S., & Wang, M. (2020). Near real-time topic-driven rumor detection in source microblogs. Knowledge-Based Systems, 207, 106391.

    Article  Google Scholar 

  31. Jwa, H., Oh, D., Park, K., Kang, J., & Lim, H. (2019). exBAKE: Automatic fake news detection model based on bidirectional encoder representations from transformers (BERT). Applied Sciences, 9(19), 4062. https://doi.org/10.3390/app9194062

    Article  Google Scholar 

  32. Reis, J. C. S., Correia, A., Murai, F., Veloso, A., & Benevenuto, F. (2019). Supervised learning for fake news detection. IEEE Intelligent Systems, 34(2), 76–81. https://doi.org/10.1109/MIS.2019.2899143

    Article  Google Scholar 

  33. “Coronavirus: The human cost of fake news in India.” https://www.bbc.com/news/world-asia-india-53165436. Accessed from 10 Mar 2022.

  34. “Fake news, rumours on social media hit Indian firms.” https://www.livemint.com/Companies/Cqbmv2eOniYHzEqLYkxFyO/Fake-news-rumours-on-social-media-hit-Indian-firms.html. Accessed from 10 Mar 2022.

  35. Wang, W. Y. (2017). “Liar, liar pants on fire": A new benchmark dataset for fake news detection,” arXiv preprintarXiv:1705.00648

  36. Patwa, P., Sharma, S., Pykl, S., Guptha, V., Kumari, G., Akhtar, M. S., Ekbal, A., Das, A., & Chakraborty, T. (2021). Fighting an Infodemic: COVID-19 fake news dataset. In International Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situation, (pp. 21–29). Springer.

    Chapter  Google Scholar 

  37. “tf–idf.” https://en.wikipedia.org/wiki/Tf%E2%80%93idf. Accessed from 12 Mar 2022.

  38. Setiawan, R., Ponnam, V. S., Sengan, S., Anam, M., Subbiah, C., Phasinam, K., Vairaven, M., & Ponnusamy, S. (2022). Certain investigation of fake news detection from facebook and twitter using artificial intelligence approach. Wireless Personal Communications, 127(2), 1737–1762. https://doi.org/10.1007/s11277-021-08720-9

    Article  Google Scholar 

  39. Pennington, J., Socher, R., Manning, C. D. (2014). “Glove: Global vectors for word representation”. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532–1543).

  40. “GloVe.” https://en.wikipedia.org/wiki/GloVe. Accessed from 12 Mar 2022.

  41. Graves, A., Schmidhuber, J. (2005). Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks 18(56), 602–610.

  42. Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE transactions on Signal Processing, 45(11), 2673–2681.

    Article  Google Scholar 

  43. “BERT Explained: State of the art language model for NLP.” https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270. Accessed from 12 Mar 2022.

  44. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., Polosukhin, I. (2017). “Attention is all you need”. Advances in Neural Information Processing Systems, 30.

  45. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K. (2018). “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprintarXiv:1810.04805

  46. “Bert-base-uncased.” https://huggingface.co/bert-base-uncased. Accessed from 12 Mar 2022.

  47. Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 267, 1–38. https://doi.org/10.1016/j.artint.2018.07.007

    Article  MathSciNet  MATH  Google Scholar 

  48. Kim, B., Khanna, R., Koyejo, O. O. (2016). “Examples are not enough, learn to criticize! criticism for interpretability”. Advances in Neural Information Processing Systems, 29.

  49. Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). “Grad-cam: Visual explanations from deep networks via gradient-based localization”. In Proceedings of the IEEE international conference on computer vision (pp. 618–626).

  50. Molnar, C. (2022). Interpretable machine learning (2 ed.) Lulu.com

  51. Doran, D., Schulz, S., & Besold, T. R. (2017). “What does explainable AI really mean? A new conceptualization of perspectives”. arXiv preprint arXiv:1710.00794

  52. Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K-R. (2021). Explaining deep neural networks and beyond: A review of methods and applications. Proceedings of the IEEE, 109(3), 247–278. https://doi.org/10.1109/JPROC.2021.3060483

    Article  Google Scholar 

  53. Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020). “A survey of the state of explainable AI for natural language processing”. arXiv preprint arXiv:2010.00711

  54. Alhindi, T., Petridis, S., & Muresan, S. (2018). “Where is your evidence: Improving fact-checking by justification modeling”. In Proceedings of the first workshop on fact extraction and verification (FEVER) (pp. 85–90).

Download references

Funding

This research did not receive any grant from any of the funding agencies. All data generated or analyzed during this study are included in this article.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Delhi, Delhi, India

    Vanshika Dua, Ankit Rajpal & Naveen Kumar

  2. Dyal Singh College, University of Delhi, Delhi, India

    Sheetal Rajpal

  3. Hansraj College, University of Delhi, Delhi, India

    Manoj Agarwal

Authors
  1. Vanshika Dua
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ankit Rajpal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sheetal Rajpal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Manoj Agarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Naveen Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ankit Rajpal.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest. This research did not receive any grant from any of the funding agencies.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dua, V., Rajpal, A., Rajpal, S. et al. I-FLASH: Interpretable Fake News Detector Using LIME and SHAP. Wireless Pers Commun 131, 2841–2874 (2023). https://doi.org/10.1007/s11277-023-10582-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-023-10582-2

Keywords

Search

Navigation