Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Multimodal PTSD characterization via the StartleMart game

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

Computer games have recently shown promise as a diagnostic and treatment tool for psychiatric rehabilitation. This paper examines the potential of combining multiple modalities for detecting affective responses of patients interacting with a simulation built on game technology, aimed at the treatment of mental diagnoses such as post traumatic stress disorder (PTSD). For that purpose, we couple game design and game technology to create a game-based tool for exposure therapy and stress inoculation training that utilizes stress detection for the automatic profiling and potential personalization of PTSD treatments. The PTSD treatment game we designed forces the player to go through various stressful experiences while a stress detection mechanism profiles the severity and type of PTSD by analyzing the physiological responses to those in-game stress elicitors in two separate modalities: skin conductance (SC) and blood volume pulse (BVP). SC is often used to monitor stress as it is connected to the activation of the sympathetic nervous system (SNS). By including BVP into the model we introduce information about para-sympathetic activation, which offers a more complete view of the psycho-physiological experience of the player; in addition, as BVP is also modulated by SNS, a multimodal model should be more robust to changes in each modality due to particular drugs or day-to-day bodily changes. Overall, the study and analysis of 14 PTSD-diagnosed veteran soldiers presented in this paper reveals correspondence between diagnostic standard measures of PTSD severity and SC and BVP responsiveness and feature combinations thereof. The study also reveals that these features are significantly correlated with subjective evaluations of the stressfulness of experiences, represented as pairwise preferences. More importantly, the results presented here demonstrate that using the modalities of SC and BVP captures a more nuanced representation of player stress responses than using SC alone. We conclude that the results support the use of the simulation as a relevant treatment tool for stress inoculation training, and suggest the feasibility of using such a tool to profile PTSD patients. The use of multiple modalities appears to be key for an accurate profiling, although further research and analysis are required to identify the most relevant physiological features for capturing user stress.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://www.wilddivine.com/.

References

  1. Acharya UR, Joseph KP, Kannathal N, Lim CM, Suri JS (2006) Heart rate variability: a review. Med Biol Eng Comput 44(12):1031–1051

    Article  Google Scholar 

  2. Andreassi JL (2000) Psychophysiology: human behavior and physiological response. Routledge, London

    Google Scholar 

  3. Benedek M, Kaernbach C (2010) Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology 47(4):647–658

    Google Scholar 

  4. Blanchard EB, Jones-Alexander J, Buckley TC, Forneris CA (1996) Psychometric properties of the PTSD checklist (PCL). Behav Res Ther 34(8):669–673

    Article  Google Scholar 

  5. Blechert J et al (2007) Autonomic and respiratory characteristics of posttraumatic stress disorder and panic disorder. Psychosom Med 69(9):935–943

    Article  Google Scholar 

  6. Boucsein W (2011) Electrodermal activity. Springer, New York

    Google Scholar 

  7. Calvo RA, D’Mello S (2010) Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans Affect Comput 1(1):18–37

    Article  Google Scholar 

  8. First MB et al (2002) Structured clinical interview for DSM-IV-TR axis I disorders, research version, patient edition (SCID-I/P). Biometrics Research Institute, New York

    Google Scholar 

  9. Foa EB et al (2009) Effective treatments for PTSD, practice guidelines from the International Society for Traumatic Stress Studies, 2nd edn. Guilford Press, New York

    Google Scholar 

  10. Goldberger J, Challapalli S, Tung R, Parker M, Kadish A (2001) Relationship of heart rate variability to parasympathetic effect. Circulation 103(15):1977

    Article  Google Scholar 

  11. Healey JA, Picard RW (2005) Detecting stress during real-world driving tasks using physiological sensors. IEEE ITS 6(2):156–166

    Google Scholar 

  12. Hernandez J, Morris RR, Picard RW (2011) Call center stress recognition with person-specific models. In: ACII. Springer, New York, pp 125–134

  13. Hoge CW et al (2004) Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care. N Engl J Med 351(1):13–22

    Article  Google Scholar 

  14. Holmgård C, Yannakakis GN, Karstoft KI, Andersen HS (2013) Stress detection for ptsd via the startlemart game. In: IEEE ACII. Springer, New York, pp 523–528

  15. Kashdan TB et al (2010) Everyday strivings in war veterans with posttraumatic stress disorder: suffering from a hyper-focus on avoidance and emotion regulation. Behav Ther 41(3):350–363

    Article  Google Scholar 

  16. Kemp AH, Quintana DS, Gray MA, Felmingham KL, Brown K, Gatt JM (2010) Impact of depression and antidepressant treatment on heart rate variability: a review and meta-analysis. Biol Psychiatry 67(11):1067–1074

    Article  Google Scholar 

  17. Kendall M (1970) Rank correlation methods, 4th edn. Griffin, London

  18. Malik M et al (1996) Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. Circulation 93(5):1043–65

    Article  Google Scholar 

  19. Magruder KM, Frueh BC, Knapp RG, Johnson MR, Vaughan JA, Carson TC, Powell DA, Hebert R (2004) PTSD symptoms, demographic characteristics, and functional status among veterans treated in VA primary care clinics. J Trauma Stress 17(4):293–301

    Article  Google Scholar 

  20. Martínez HP, Garbarino M, Yannakakis GN (2011) Generic physiological features as predictors of player experience. In: ACII. Springer, New York, pp 267–276

  21. Martinez HP, Bengio Y, Yannakakis GN (2013) Learning deep physiological models of affect. IEEE Comput Intell Mag 8(2):20–33

    Article  Google Scholar 

  22. Moody G (1993) Spectral analysis of heart rate without resampling. In: IEEE proceedings of the computers in cardiology, pp 715–718

  23. Perala C (2007) Galvanic skin response as a measure of soldier stress. Tech. rep, DTIC document

  24. Picard R, Vyzas E, Healey J (2001) Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans Pattern Anal Mach Intell 23(10):1175–1191

  25. Picard RW, Vyzas E, Healey J (2001) Toward machine emotional intelligence: analysis of affective physiological state. IEEE PAMI 23(10):1175–1191

  26. Pole N (2007) The psychophysiology of posttraumatic stress disorder: a meta-analysis. Psychol Bull 133(5):725

  27. Popović S, Horvat M, Kukolja D, Dropuljić B, Cosić K (2009) Stress inoculation training supported by physiology-driven adaptive virtual reality stimulation. Stud Health Technol Inform 144:50–54. doi:10.3233/978-1-60750-017-9-50

  28. Robinson MD, Clore GL (2002) Belief and feeling: ervidence for an accessibility model of emotional self-report. Psychol Bull 128(6):934

    Article  Google Scholar 

  29. Siepmann M et al (2003) Effects of sertraline on autonomic and cognitive functions in healthy volunteers. Psychopharmacology 168(3):293–298

    Article  Google Scholar 

  30. Tognetti S et al (2011) The affective triad: stimuli, questionnaires, and measurements. In: ACII. Springer, New York, pp 101–110

  31. Wolpe J (1973) The practice of behavior therapy. Pergamon Press, New York

    Google Scholar 

  32. Wood DP et al (2011) Reality graded exposure therapy with physiological monitoring for the treatment of combat related post traumatic stress disorder: a pilot study. Stud Health Technol Inform 163:696

    Google Scholar 

  33. Yannakakis G, Hallam J (2008) Entertainment modeling through physiology in physical play. Int J Hum Comput Stud 66(10):741–755

  34. Yannakakis GN, Hallam J (2008) Entertainment modeling through physiology in physical play. Int J Hum Comput Stud 66(10):741–755

  35. Yannakakis GN, Hallam J (2008) Entertainment modeling through physiology in physical play. Int J Hum Comput Stud 66(10):741–755

  36. Yannakakis GN, Hallam J (2011) Ranking vs. preference: a comparative study of self-reporting. In: ACII. Springer, New York, pp 437–446

  37. Zhai J et al (2005) Realization of stress detection using psychophysiological signals for improvement of human–computer interactions. In: Proceedings of the IEEE SoutheastCon. IEEE, pp 415–420

Download references

Acknowledgments

This research was supported by the Danish Council for Technology and Innovation under the Games for Health project and by the FP7 ICT project ILearnRW (Project No: 318803).

Author information

Authors and Affiliations

  1. Center for Computer Games Research, IT University of Copenhagen, Copenhagen, Denmark

    Christoffer Holmgård

  2. Institute for Digital Games, University of Malta, Msida, Malta

    Georgios N. Yannakakis & Héctor P. Martínez

  3. National Center for Psychotraumatology, University of Southern Denmark, Odense, Denmark

    Karen-Inge Karstoft

  4. Centre for Crisis and Disaster Psychiatry, Mental Health Services of the Capital Region of Denmark, Copenhagen, Denmark

    Henrik Steen Andersen

Authors
  1. Christoffer Holmgård
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Georgios N. Yannakakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Héctor P. Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Karen-Inge Karstoft
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Henrik Steen Andersen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christoffer Holmgård.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Holmgård, C., Yannakakis, G.N., Martínez, H.P. et al. Multimodal PTSD characterization via the StartleMart game. J Multimodal User Interfaces 9, 3–15 (2015). https://doi.org/10.1007/s12193-014-0160-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-014-0160-5

Keywords

Search

Navigation