research-article

SenseMood: Depression Detection on Social Media

Authors:

Henry LeungAuthors Info & Claims

ICMR '20: Proceedings of the 2020 International Conference on Multimedia Retrieval

Pages 407 - 411

https://doi.org/10.1145/3372278.3391932

Published: 08 June 2020 Publication History

Abstract

More than 300 million people have been affected by depression all over the world. Due to the medical equipment and knowledge limitations, most of them are not diagnosed at the early stages. Recent work attempts to use social media to detect depression since the patterns of opinions and thoughts expression of the posted text and images, can reflect users' mental state to some extent. In this work, we design a system dubbed SenseMood to demonstrate that the users with depression can be efficiently detected and analyzed by using proposed system. A deep visual-textual multimodal learning approach has been proposed to reveal the psychological state of the users on social networks. The posted images and tweets data from users with/without depression on Twitter have been collected and used for depression detection. CNN-based classifier and Bert are applied to extract the deep features from the pictures and text posted by users respectively. Then visual and textual features are combined to reflect the emotional expression of users. Finally our system classifies the users with depression and normal users through a neural network and the analysis report is generated automatically.

References

[1]

Oluwaseun Ajao, Deepayan Bhowmik, and Shahrzad Zargari. 2019. Sentiment aware fake news detection on online social networks. In ICASSP 2019--2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2507--2511.

[2]

Tuka Al Hanai, Mohammad M Ghassemi, and James R Glass. 2018. Detecting Depression with Audio/Text Sequence Modeling of Interviews. In Interspeech. 1716--1720.

[3]

Munmun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. 2013. Predicting depression via social media. In Seventh international AAAI conference on weblogs and social media.

[4]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[5]

Stephanie Gillespie, Elliot Moore, Jacqueline Laures-Gore, Matthew Farina, Scott Russell, and Yash-Yee Logan. 2017. Detecting stress and depression in adults with aphasia through speech analysis. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 5140--5144.

Digital Library

[6]

Pengwei Hu, Tiantian He, Keith CC Chan, and Henry Leung. 2017. Deep Fusion of Multiple Networks for Learning Latent Social Communities. In 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI). IEEE, 765--771.

[7]

Pengwei Hu, Zhaomeng Niu, Tiantian He, and Keith CC Chan. 2018. Learning Deep Representations in Large Integrated Network for Graph Clustering. In 2018 IEEE First International Conference on Artificial Intelligence and Knowledge Engineering (AIKE). IEEE, 101--105.

[8]

Po-Yao Huang, Junwei Liang, Jean-Baptiste Lamare, and Alexander G Hauptmann. 2018. Multimodal filtering of social media for temporal monitoring and event analysis. In Proceedings of the 2018 ACMon International Conference on Multimedia Retrieval. 450--457.

Digital Library

[9]

Felice N Jacka and Nicola J Reavley. 2014. Prevention of mental disorders: evidence, challenges and opportunities.

[10]

Kurt Kroenke, TaraWStrine, Robert L Spitzer, Janet BW Williams, Joyce T Berry, and Ali H Mokdad. 2009. The PHQ-8 as a measure of current depression in the general population. Journal of affective disorders 114, 1--3 (2009), 163--173.

[11]

Genevieve Lam, Huang Dongyan, and Weisi Lin. 2019. Context-aware Deep Learning for Multi-modal Depression Detection. In ICASSP 2019--2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 3946--3950.

[12]

Chenhao Lin and Ajay Kumar. 2018. A CNN-based framework for comparison of contactless to contact-based fingerprints. IEEE Transactions on Information Forensics and Security 14, 3 (2018), 662--676.

[13]

Zhun Liu, Ying Shen, Varun Bharadhwaj Lakshminarasimhan, Paul Pu Liang, Amir Zadeh, and Louis-Philippe Morency. 2018. Efficient low-rank multimodal fusion with modality-specific factors. arXiv preprint arXiv:1806.00064 (2018).

[14]

Chandler May, Alex Wang, Shikha Bordia, Samuel R Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. arXiv preprint arXiv:1903.10561 (2019).

[15]

Seungwhan Moon, Leonardo Neves, and Vitor Carvalho. 2018. Multimodal named entity disambiguation for noisy social media posts. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2000--2008.

[16]

Ricardo F Muñoz, William R Beardslee, and Yan Leykin. 2012. Major depression can be prevented. American Psychologist 67, 4 (2012), 285.

[17]

Ricardo F Munoz, Patricia J Mrazek, and Robert J Haggerty. 1996. Institute of Medicine report on prevention of mental disorders: summary and commentary. American Psychologist 51, 11 (1996), 1116.

[18]

Hyeonseob Nam, Jung-Woo Ha, and Jeonghee Kim. 2017. Dual attention networks for multimodal reasoning and matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 299--307.

[19]

Minsu Park, DavidWMcDonald, and Meeyoung Cha. 2013. Perception differences between the depressed and non-depressed users in twitter. In Seventh International AAAI Conference on Weblogs and Social Media.

[20]

A Picardi, I Lega, L Tarsitani, M Caredda, G Matteucci, MP Zerella, R Miglio, A Gigantesco, M Cerbo, A Gaddini, et al. 2016. A randomised controlled trial of the effectiveness of a program for early detection and treatment of depression in primary care. Journal of affective disorders 198 (2016), 96--101.

[21]

Andrew G Reece and Christopher M Danforth. 2017. Instagram photos reveal predictive markers of depression. EPJ Data Science 6, 1 (2017), 15.

[22]

Guangyao Shen, Jia Jia, Liqiang Nie, Fuli Feng, Cunjun Zhang, Tianrui Hu, Tat- Seng Chua, and Wenwu Zhu. 2017. Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution. In IJCAI. 3838--3844.

[23]

Tiancheng Shen, Jia Jia, Guangyao Shen, Fuli Feng, Xiangnan He, Huanbo Luan, Jie Tang, Thanassis Tiropanis, Tat-Seng Chua, and Wendy Hall. 2018. Cross- Domain Depression Detection via Harvesting Social Media. In IJCAI. 1611--1617.

[24]

Melissa N Stolar, Margaret Lech, and Nicholas B Allen. 2015. Detection of depression in adolescents based on statistical modeling of emotional influences in parent-adolescent conversations. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 987--991.

[25]

Andrew Yates, Arman Cohan, and Nazli Goharian. 2017. Depression and self harm risk assessment in online forums. arXiv preprint arXiv:1709.01848 (2017).

[26]

Shumei Zhang, Jia Jia, and Yishuang Ning. 2017. Inferring emotions from heterogeneous social media data: A Cross-media Auto-Encoder solution. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2891--2895.

Digital Library

Cited By

Khoo LLim MChong CMcNaney R(2024)Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing ApproachesSensors10.3390/s2402034824:2(348)Online publication date: 6-Jan-2024
https://doi.org/10.3390/s24020348
Salas-Zárate RAlor-Hernández GParedes-Valverde MSalas-Zárate MBustos-López MSánchez-Cervantes J(2024)Mental-Health: An NLP-Based System for Detecting Depression Levels through User Comments on Twitter (X)Mathematics10.3390/math1213192612:13(1926)Online publication date: 21-Jun-2024
https://doi.org/10.3390/math12131926
Hu PHu LWang FMei J(2024)Editorial: Computing and artificial intelligence in digital therapeuticsFrontiers in Medicine10.3389/fmed.2023.133068610Online publication date: 5-Jan-2024
https://doi.org/10.3389/fmed.2023.1330686
Show More Cited By

Index Terms

SenseMood: Depression Detection on Social Media
1. Applied computing
  1. Life and medical sciences
2. Information systems
  1. Information retrieval
    1. Users and interactive retrieval

Recommendations

Improving Depression Classification in Social Media Text with Transformer Ensembles
SOICT '23: Proceedings of the 12th International Symposium on Information and Communication Technology

Depression is an emotional disorder and it can arise in each of us individually. In the era of the Fourth Industrial Revolution, social media platforms provide an opportunity for people to share their thoughts and feelings, which may reveal early signs ...
Harnessing emotions for depression detection
Abstract
Human emotions using textual cues, speech patterns, and facial expressions can give insight into their mental state. Although there are several uni-modal datasets for emotion recognition, there are very few labeled datasets for multi-modal ...
Exploring Social Media for Early Detection of Depression in COVID-19 Patients
WWW '23: Proceedings of the ACM Web Conference 2023

The COVID-19 pandemic has caused substantial damage to global health. Even though three years have passed, the world continues to struggle with the virus. Concerns are growing about the impact of COVID-19 on the mental health of infected individuals, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '20: Proceedings of the 2020 International Conference on Multimedia Retrieval

June 2020

605 pages

ISBN:9781450370875

DOI:10.1145/3372278

General Chairs:
Cathal Gurrin
Dublin City University, Ireland
,
Björn Þór Jónsson
IT University of Copenhagen, Denmark
,
Noriko Kando
National Institute of Informatics, Tokyo
,
Program Chairs:
Klaus Schoeffmann
Klagenfurt University, Austria
,
Phoebe Chen
La Trobe University, Australia
,
Noel E. O'Connor
Dublin City University, Ireland

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICMR '20

Sponsor:

SIGMM

ICMR '20: International Conference on Multimedia Retrieval

June 8 - 11, 2020

Dublin, Ireland

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

81
Total Citations
View Citations
1,581
Total Downloads

Downloads (Last 12 months)365
Downloads (Last 6 weeks)40

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Khoo LLim MChong CMcNaney R(2024)Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing ApproachesSensors10.3390/s2402034824:2(348)Online publication date: 6-Jan-2024
https://doi.org/10.3390/s24020348
Salas-Zárate RAlor-Hernández GParedes-Valverde MSalas-Zárate MBustos-López MSánchez-Cervantes J(2024)Mental-Health: An NLP-Based System for Detecting Depression Levels through User Comments on Twitter (X)Mathematics10.3390/math1213192612:13(1926)Online publication date: 21-Jun-2024
https://doi.org/10.3390/math12131926
Hu PHu LWang FMei J(2024)Editorial: Computing and artificial intelligence in digital therapeuticsFrontiers in Medicine10.3389/fmed.2023.133068610Online publication date: 5-Jan-2024
https://doi.org/10.3389/fmed.2023.1330686
Garg M(2024)Towards Mental Health Analysis in Social Media for Low-resourced LanguagesACM Transactions on Asian and Low-Resource Language Information Processing10.1145/363876123:3(1-22)Online publication date: 9-Mar-2024
https://dl.acm.org/doi/10.1145/3638761
Cabral RLuo SPoon JHan SSerra ESpezzano F(2024)3M-Health: Multimodal Multi-Teacher Knowledge Distillation for Mental Health DetectionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679635(152-162)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679635
Anshul APranav GRehman MKumar N(2024)A Multimodal Framework for Depression Detection During COVID-19 via Harvesting Social MediaIEEE Transactions on Computational Social Systems10.1109/TCSS.2023.330922911:2(2872-2888)Online publication date: Apr-2024
https://doi.org/10.1109/TCSS.2023.3309229
Saraswat PBeniwal R(2024)BERT-based RNN for Effective Detection of Depression with Severity Levels from Text Data2024 IEEE Symposium on Wireless Technology & Applications (ISWTA)10.1109/ISWTA62130.2024.10651873(52-56)Online publication date: 20-Jul-2024
https://doi.org/10.1109/ISWTA62130.2024.10651873
Balcı ESaraç E(2024)Automated Depression Detection from Tweets: a Comparison of NLP Techniques2024 8th International Artificial Intelligence and Data Processing Symposium (IDAP)10.1109/IDAP64064.2024.10711029(1-5)Online publication date: 21-Sep-2024
https://doi.org/10.1109/IDAP64064.2024.10711029
Zafar AAftab DQureshi RWang YYan H(2024)Multi-Explainable TemporalNet: An Interpretable Multimodal Approach using Temporal Convolutional Network for User-level Depression Detection2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00231(2258-2265)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00231
Akter Asma SAkhter NSharmin SRahman MSanwar Hosen ALee ORa I(2024)Hierarchical Explainable Network for Investigating Depression From Multilingual Textual DataIEEE Access10.1109/ACCESS.2024.345881512(131915-131927)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3458815
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents