research-article

Topic Modeling Based Multi-modal Depression Detection

Authors:

Christian PoellabauerAuthors Info & Claims

AVEC '17: Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge

Pages 69 - 76

https://doi.org/10.1145/3133944.3133945

Published: 23 October 2017 Publication History

Abstract

Major depressive disorder is a common mental disorder that affects almost 7% of the adult U.S. population. The 2017 Audio/Visual Emotion Challenge (AVEC) asks participants to build a model to predict depression levels based on the audio, video, and text of an interview ranging between 7-33 minutes. Since averaging features over the entire interview will lose most temporal information, how to discover, capture, and preserve useful temporal details for such a long interview are significant challenges. Therefore, we propose a novel topic modeling based approach to perform context-aware analysis of the recording. Our experiments show that the proposed approach outperforms context-unaware methods and the challenge baselines for all metrics.

References

[1]

Tadas Baltruvšaitis, Peter Robinson, and Louis-Philippe Morency. 2016. Openface: an open source facial behavior analysis toolkit Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on. IEEE, 1--10.

[2]

David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research Vol. 3, Jan (2003), 993--1022.

Digital Library

[3]

Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N. Chang, Sungbok Lee, and Shrikanth S. Narayanan. 2008. IEMOCAP: Interactive emotional dyadic motion capture database. Language resources and evaluation Vol. 42, 4 (2008), 335.

[4]

Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. 2014. COVAREP: A collaborative voice analysis repository for speech technologies Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on. IEEE, 960--964.

[5]

David DeVault, Ron Artstein, Grace Benn, Teresa Dey, Ed Fast, Alesia Gainer, Kallirroi Georgila, Jon Gratch, Arno Hartholt, Margaux Lhommet, et al. 2014. SimSensei Kiosk: A virtual human interviewer for healthcare decision support Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems. International Foundation for Autonomous Agents and Multiagent Systems, 1061--1068.

Digital Library

[6]

Maurizio Fava and Kenneth S Kendler. 2000. Major depressive disorder. Neuron, Vol. 28, 2 (2000), 335--341.

[7]

Jonathan Gratch, Ron Artstein, Gale M. Lucas, Giota Stratou, Stefan Scherer, Angela Nazarian, Rachel Wood, Jill Boberg, David DeVault, Stacy Marsella, et al. 2014. The Distress Analysis Interview Corpus of human and computer interviews LREC. 3123--3128.

[8]

MA Hall. 1998. Correlation-based feature subset selection for machine learning. Thesis submitted in partial fulfillment of the requirements of the degree of Doctor of Philosophy at the University of Waikato (1998).

[9]

Liangjie Hong and Brian D. Davison. 2010. Empirical study of topic modeling in twitter. In Proceedings of the first workshop on social media analytics. ACM, 80--88.

Digital Library

[10]

Kurt Kroenke, Tara W. Strine, Robert L. Spitzer, Janet B. W. Williams, Joyce T. Berry, and Ali H. Mokdad. 2009. The PHQ-8 as a measure of current depression in the general population. Journal of affective disorders Vol. 114, 1 (2009), 163--173.

[11]

Xu-Ying Liu, Jianxin Wu, and Zhi-Hua Zhou. 2009. Exploratory undersampling for class-imbalance learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), Vol. 39, 2 (2009), 539--550.

Digital Library

[12]

Qiaozhu Mei, Deng Cai, Duo Zhang, and ChengXiang Zhai. 2008. Topic modeling with network regularization. In Proceedings of the 17th international conference on World Wide Web. ACM, 101--110.

Digital Library

[13]

Md Nasir, Arindam Jati, Prashanth Gurunath Shivakumar, Sandeep Nallan Chakravarthula, and Panayiotis Georgiou. 2016. Multimodal and multiresolution depression detection from speech and facial landmark features Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge. ACM, 43--50.

Digital Library

[14]

Anastasia Pampouchidou, Olympia Simantiraki, Amir Fazlollahi, Matthew Pediaditis, Dimitris Manousos, Alexandros Roniotis, Georgios Giannakakis, Fabrice Meriaudeau, Panagiotis Simos, Kostas Marias, et al. 2016. Depression Assessment by Fusing High and Low Level Features from Audio, Video, and Text Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge. ACM, 27--34.

Digital Library

[15]

James W. Pennebaker, Ryan L. Boyd, Kayla Jordan, and Kate Blackburn. 2015. The development and psychometric properties of LIWC2015. Technical Report.

[16]

Philip Resnik, Anderson Garron, and Rebecca Resnik. 2013. Using topic modeling to improve prediction of neuroticism and depression Proceedings of the 2013 Conference on Empirical Methods in Natural. Association for Computational Linguistics, 1348--1353.

[17]

Fabien Ringeval, Björn Schuller, Michel Valstar, Jonathan Gratch, Roddy Cowie, Stefan Scherer, Sharon Mozgai, Nicholas Cummins, Maximilian Schmitt, and Maja Pantic. 2017. AVEC 2017: Real-life Depression, and Affect Recognition Workshop and Challenge Proceedings of the 7th International Workshop on Audio/Visual Emotion Challenge. ACM, 1--8.

Digital Library

[18]

JAN Spijker, Ron De Graaf, Rob V. Bijl, Aartjan T. F. Beekman, Johan Ormel, and Willem A. Nolen. 2002. Duration of major depressive episodes in the general population: results from The Netherlands Mental Health Survey and Incidence Study (NEMESIS). The British journal of psychiatry Vol. 181, 3 (2002), 208--213.

[19]

Michel Valstar, Jonathan Gratch, Björn Schuller, Fabien Ringeval, Dennis Lalanne, Mercedes Torres Torres, Stefan Scherer, Giota Stratou, Roddy Cowie, and Maja Pantic. 2016. Avec 2016: Depression, mood, and emotion recognition workshop and challenge Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge. ACM, 3--10.

Digital Library

[20]

Chong Wang and David M. Blei. 2011. Collaborative topic modeling for recommending scientific articles Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 448--456.

Digital Library

[21]

James R. Williamson, Elizabeth Godoy, Miriam Cha, Adrianne Schwarzentruber, Pooya Khorrami, Youngjune Gwon, Hsiang-Tsung Kung, Charlie Dagli, and Thomas F. Quatieri. 2016. Detecting Depression using Vocal, Facial and Semantic Communication Cues Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge. ACM, 11--18.

Digital Library

[22]

Le Yang, Dongmei Jiang, Lang He, Ercheng Pei, Meshia Cédric Oveneke, and Hichem Sahli. 2016. Decision Tree Based Depression Classification from Audio Video and Language Information Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge. ACM, 89--96.

Digital Library

Cited By

Krishna NVikas Reddy RLikhith MPriya N(2024)Tackling Depression Detection With Deep LearningDriving Smart Medical Diagnosis Through AI-Powered Technologies and Applications10.4018/979-8-3693-3679-3.ch006(102-117)Online publication date: 9-Feb-2024
https://doi.org/10.4018/979-8-3693-3679-3.ch006
Khoo LLim MChong CMcNaney R(2024)Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing ApproachesSensors10.3390/s2402034824:2(348)Online publication date: 6-Jan-2024
https://doi.org/10.3390/s24020348
Zeulner THagerer GMüller MVazquez IGloor P(2024)Predicting Individual Well-Being in Teamwork Contexts Based on Speech FeaturesInformation10.3390/info1504021715:4(217)Online publication date: 12-Apr-2024
https://doi.org/10.3390/info15040217
Show More Cited By

Index Terms

Topic Modeling Based Multi-modal Depression Detection
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
2. Information systems
  1. Information retrieval
    1. Document representation
      1. Document topic models
    2. Specialized information retrieval
      1. Multimedia and multimodal retrieval

Recommendations

Multi-Modal Depression Detection Based on High-Order Emotional Features
AICCC '22: Proceedings of the 2022 5th Artificial Intelligence and Cloud Computing Conference

The diagnosis of depression has always been a difficulty in its treatment. At present, the research on automatic depression detection mostly directly uses low-order features such as video, audio and text as input. The lack of guidance of high-order ...
Transition to Adulthood for Young People with Intellectual or Developmental Disabilities: Emotion Detection and Topic Modeling
Social, Cultural, and Behavioral Modeling
Abstract
Transition to Adulthood is an essential life stage for many families. The prior research has shown that young people with intellectual or development disabilities (IDD) have more challenges than their peers. This study is to explore how to use ...
Topic modeling in software engineering research
Abstract
Topic modeling using models such as Latent Dirichlet Allocation (LDA) is a text mining technique to extract human-readable semantic “topics” (i.e., word clusters) from a corpus of textual documents. In software engineering, topic modeling has been ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

AVEC '17: Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge

October 2017

78 pages

ISBN:9781450355025

DOI:10.1145/3133944

General Chairs:
Fabien Ringeval
University of Grenoble Alpes
,
Björn Schuller
University of Passau/Imperial College London
,
Michel Valstar
University of Nottingham
,
Jonathan Gratch
University of Southern California
,
Roddy Cowie
Queen's University Belfast
,
Maja Pantic
Imperial College London/Twente University, UK/The Netherlands

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '17

Sponsor:

SIGMM

MM '17: ACM Multimedia Conference

October 23, 2017

California, Mountain View, USA

Acceptance Rates

AVEC '17 Paper Acceptance Rate 8 of 17 submissions, 47%;

Overall Acceptance Rate 52 of 98 submissions, 53%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

81
Total Citations
View Citations
1,383
Total Downloads

Downloads (Last 12 months)163
Downloads (Last 6 weeks)8

Reflects downloads up to 02 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Krishna NVikas Reddy RLikhith MPriya N(2024)Tackling Depression Detection With Deep LearningDriving Smart Medical Diagnosis Through AI-Powered Technologies and Applications10.4018/979-8-3693-3679-3.ch006(102-117)Online publication date: 9-Feb-2024
https://doi.org/10.4018/979-8-3693-3679-3.ch006
Khoo LLim MChong CMcNaney R(2024)Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing ApproachesSensors10.3390/s2402034824:2(348)Online publication date: 6-Jan-2024
https://doi.org/10.3390/s24020348
Zeulner THagerer GMüller MVazquez IGloor P(2024)Predicting Individual Well-Being in Teamwork Contexts Based on Speech FeaturesInformation10.3390/info1504021715:4(217)Online publication date: 12-Apr-2024
https://doi.org/10.3390/info15040217
Vashchenko V(2024)Topic modeling for short texts: comparative analysis of algorithmsSociology: methodology, methods, mathematical modeling (Sociology: 4M)10.19181/4m.2023.32.1.229:56(69-112)Online publication date: 2024
https://doi.org/10.19181/4m.2023.32.1.2
Jianhua TCunhang FZheng LZhao LYing SShan L(2024)Development of multimodal sentiment recognition and understandingJournal of Image and Graphics10.11834/jig.24001729:6(1607-1627)Online publication date: 2024
https://doi.org/10.11834/jig.240017
Nepal SPillai AWang WGriffin TCollins AHeinz MLekkas DMirjafari SNemesure MPrice GJacobson NCampbell A(2024)MoodCapture: Depression Detection using In-the-Wild Smartphone ImagesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642680(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642680
Li MSun XWang M(2024)Detecting Depression With Heterogeneous Graph Neural Network in Clinical Interview TranscriptIEEE Transactions on Computational Social Systems10.1109/TCSS.2023.326305611:1(1315-1324)Online publication date: Feb-2024
https://doi.org/10.1109/TCSS.2023.3263056
Guo YLiu JWang LQin WHao SHong R(2024)A Prompt-Based Topic-Modeling Method for Depression Detection on Low-Resource DataIEEE Transactions on Computational Social Systems10.1109/TCSS.2023.326008011:1(1430-1439)Online publication date: Feb-2024
https://doi.org/10.1109/TCSS.2023.3260080
Hassan ABernadin S(2024)A Comprehensive Analysis of Speech Depression Recognition SystemsSoutheastCon 202410.1109/SoutheastCon52093.2024.10500078(1509-1518)Online publication date: 15-Mar-2024
https://doi.org/10.1109/SoutheastCon52093.2024.10500078
Liang KChen JHe TWang WSingh ARawat DSong HLyu Z(2024)Review of the Open Data Sets for Contactless SensingIEEE Internet of Things Journal10.1109/JIOT.2024.335183811:11(19000-19022)Online publication date: 1-Jun-2024
https://doi.org/10.1109/JIOT.2024.3351838
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents