Abstract
Depression is a genuine medical condition characterized by lethargy, suicidal thoughts, trouble concentrating, and a general state of disarray. It is a “biological brain disorder” and a psychological state of mind. The World Health Organization (WHO) estimates that over 280 million people worldwide suffer from depression, regardless of their culture, caste, religion, or whereabouts. Depression affects how a person thinks, speaks, or communicates with the outside world. The key objective of this study was to try to identify and use those differences in linguistics in Reddit posts to determine if a person may suffer from depressive disorders. This paper proposes novel Natural Language Processing (NLP) techniques, and Machine Learning approaches to train and evaluate the models. The proposed textual context-aware depression detection methodology consists of a hybrid transformer network consisting of Bidirectional Encoder Representations from Transformers (BERT) and Bidirectional Long Short-Term Memory (Bi-LSTM) with a Multi Layered Perceptron (MLP) attached in the end to classify depression indicative texts that can achieve incredible results in terms of accuracy–0.9548, precision–0.9706, recall–0.9745 and F1 score–0.9725.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
WHO International (2022) Mental health: strengthening our response. Retrieved from https://www.who.int/news-room/fact-sheets/detail/mental-health-strengthening-our-response. Accessed on 20 Jul 2022
American Psychiatric Association (2022) What is depression?. Retrieved from https://www.psychiatry.org/patients-families/depression/what-is-depression. Accessed on 20 Jul 2022
WHO International (2022) Depression. Retrieved from https://www.who.int/health-topics/depression#tab=tab_1. Accessed on 20 Jul 2022
National Institute of Mental Health (NIMH) (2022) Depression. Retrieved from https://www.nimh.nih.gov/health/topics/depression. Accessed on 20 Jul 2022
Havigerová JM, Haviger J, Kučera D, Hoffmannová P (2019) Text-based detection of the risk of depression. Front Psychol 10:513. https://doi.org/10.3389/fpsyg.2019.00513
Tadesse MM, Lin H, Xu B, Yang L (2019) Detection of depression-related posts in Reddit social media forum. IEEE Access 7:44883–44893. https://doi.org/10.1109/ACCESS.2019.2909180
Orabi AH, Buddhitha P, Orabi MH, Inkpen D (2018) Deep learning for depression detection of twitter users. In: Proceedings of the fifth workshop on computational linguistics and clinical psychology: from keyboard to clinic, pp 88–97. https://doi.org/10.18653/v1/W18-0609
Resnik P, Armstrong W, Claudino L, Nguyen T, Nguyen V-A, Boyd-Graber J (2015) Beyond LDA: exploring supervised topic modeling for depression-related language in Twitter. In: Proceedings of the 2nd workshop on computational linguistics and clinical psychology: from linguistic signal to clinical reality, pp 99–107. https://doi.org/10.3115/v1/W15-1212
Pirina I, Çöltekin Ç (2018) Identifying depression on Reddit: The effect of training data. In: Proceedings of the 2018 EMNLP workshop SMM4H: the 3rd social media mining for health applications workshop & shared task, pp 9–12. https://doi.org/10.18653/v1/W18-5903
Shen T, Jia J, Shen G, Feng F, He X, Luan H, Tang J, Tiropanis T, Chua TS, Hall W (2018) Cross-domain depression detection via harvesting social media. In: Proceedings of the 27th international joint conference on artificial intelligence (IJCAI 2018), vol 2018-july. International Joint Conferences on Artificial Intelligence, pp 1611–1617. https://doi.org/10.24963/ijcai.2018/223
Paul S, Jandhyala SK, Basu T (2018) Early detection of signs of anorexia and depression oversocial media using effective machine learning frameworks. In: Proceedings of the CLEF (working notes), pp 10–14
Wolohan JT, Hiraga M, Mukherjee A, Sayyed ZA, Millard M (2018) Detecting linguistic traces of depression in topic-restricted text: attending to self-stigmatized depression with NLP. In: Proceedings of the first international workshop on language cognition and computational models, pp 11–21
Maupomé D, Meurs M-J (2018) Using topic extraction on social media content for the early detection of depression. In: CLEF (working notes), vol 2125
Reece AG, Reagan AJ, Lix KLM, Dodds PS, Danforth CM, Langer EJ (2017) Forecasting the onset and course of mental illness with Twitter data. Sci Rep 7(1):1–11. https://doi.org/10.1038/s41598-017-12961-9
Tsugawa S, Kikuchi Y, Kishino F, Nakajima K, Itoh Y, Ohsaki H (2015) Recognizing depression from twitter activity. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems (CHI’15). Association for Computing Machinery, New York, NY, USA, pp 3187–3196. https://doi.org/10.1145/2702123.2702280
Guntuku SC, Yaden DB, Kern ML, Ungar LH, Eichstaedt JC (2017) Detecting depression and mental illness on social media: an integrative review. Curr Opin Behav Sci 18:43–49 ISSN 2352–1546. https://doi.org/10.1016/j.cobeha.2017.07.005.
Nguyen T, Phung D, Dao B, Venkatesh S, Berk M (2014) Affective and content analysis of online depression communities. IEEE Trans Affect Comp 5(3):217–226.https://doi.org/10.1109/TAFFC.2014.2315623
Tyshchenko Y (2018) Depression and anxiety detection from blog posts data. Master’s thesis, University of Tartu
Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805, arXiv:1810.04805v2. https://doi.org/10.48550/arXiv.1810.04805
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, vol 30, pp 5998–6008. arXiv:1706.03762, arXiv:1706.03762v5. https://doi.org/10.48550/arXiv.1706.03762
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Houlsby N et al (2020) An image is worth 16×16 words: transformers for image recognition at scale. arXiv:2010.11929, arXiv:2010.11929v2. https://doi.org/10.48550/arXiv.2010.11929
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Daru, D., Surani, H., Koladia, H., Parmar, K., Srivastava, K. (2023). Depression Detection Using Hybrid Transformer Networks. In: Sharma, N., Goje, A., Chakrabarti, A., Bruckstein, A.M. (eds) Data Management, Analytics and Innovation. ICDMAI 2023. Lecture Notes in Networks and Systems, vol 662. Springer, Singapore. https://doi.org/10.1007/978-981-99-1414-2_44
Download citation
DOI: https://doi.org/10.1007/978-981-99-1414-2_44
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-1413-5
Online ISBN: 978-981-99-1414-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)