research-article

Towards a Computational Framework for Automating Substance Use Counseling with Virtual Agents

Authors:

Stefan Olafsson,

Timothy BickmoreAuthors Info & Claims

AAMAS '20: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems

Pages 966 - 974

Published: 13 May 2020 Publication History

Abstract

Motivational interviewing is a counseling technique that involves the in-depth exploration of a person's reasons for and against changing their behavior, and is particularly effective for substance use counseling. We are developing a computational framework that uses techniques from motivational interviewing to conduct substance use counseling sessions by simulating face-to-face interactions with a virtual agent. We evaluated the feasibility of using a virtual agent system that uses a constrained-input modality and dialogue trees to automate parts of motivational interviewing, and report the results conducted with patients at two substance use treatment facilities. We are extending this prototype to encompass all of motivational interviewing by processing information from unconstrained user speech. To that end, we report results from training a dialog act prediction model on 132 transcripts of patient-provider counseling sessions. Our best model realized an F1 score of 0.62, recall of 0.61, precision of 0.65 and accuracy of 0.6 across five classes. This indicates reasonably good performance, highlighting the potential of this approach.

References

[1]

Timothy W Bickmore, Daniel Schulman, and Candace L Sidner. 2011. A reusable framework for health counseling dialogue systems based on a behavioral medicine ontology. Journal of biomedical informatics, Vol. 44, 2 (2011), 183--197.

Digital Library

[2]

Sarah Bowen, Katie Witkiewitz, Seema L Clifasefi, Joel Grow, Neharika Chawla, Sharon H Hsu, Haley A Carroll, Erin Harrop, Susan E Collins, M Kathleen Lustyk, et almbox. 2014. Relative efficacy of mindfulness-based relapse prevention, standard relapse prevention, and treatment as usual for substance use disorders: a randomized clinical trial. JAMA psychiatry, Vol. 71, 5 (2014), 547--556.

[3]

Justine Cassell, Joseph Sullivan, Elizabeth Churchill, and Scott Prevost. 2000. Embodied conversational agents. MIT press.

[4]

Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. (2014). arxiv: cs.CL/1406.1078

[5]

Herbert H Clark, Susan E Brennan, et almbox. 1991. Grounding in communication. Perspectives on socially shared cognition, Vol. 13, 1991 (1991), 127--149.

[6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805 (2018).

[7]

James Gibson, Dogan Can, Bo Xiao, Zac E Imel, David C Atkins, Panayiotis Georgiou, and Shrikanth Narayanan. 2016. A deep learning approach to modeling empathy in addiction counseling. Commitment, Vol. 111 (2016), 21.

[8]

Mehedi Hasan, April Idalski Carcone, Sylvie Naar, Susan Eggly, Gwen L Alexander, Kathryn E Brogan Hartlieb, and Alexander Kotov. 2019. Identifying effective motivational interviewing communication sequences using automated pattern analysis. Journal of Healthcare Informatics Research, Vol. 3, 1 (2019), 86--106.

[9]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.

[10]

Christopher W Kahler, Amy J Caswell, M Barton Laws, Justin Walthers, Molly Magill, Nadine R Mastroleo, Chanelle J Howe, Timothy Souza, Ira Wilson, Kendall Bryant, et almbox. 2016. Using topic coding to understand the nature of change language in a motivational intervention to reduce alcohol and sex risk behaviors in emergency department patients. Patient education and counseling, Vol. 99, 10 (2016), 1595--1602.

[11]

Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858 (2019).

[12]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. (2014). arxiv: cs.LG/1412.6980

[13]

John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. (2001).

[14]

Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In International conference on machine learning. 1188--1196.

Digital Library

[15]

Christine Lisetti, Reza Amini, and Ugan Yasavur. 2015. Now all together: overview of virtual health assistants emulating face-to-face health interview experience. KI-Künstliche Intelligenz, Vol. 29, 2 (2015), 161--172.

[16]

Christine Lisetti, Reza Amini, Ugan Yasavur, and Naphtali Rishe. 2013. I can help you change! an empathic virtual agent delivers behavior change health interventions. ACM Transactions on Management Information Systems (TMIS), Vol. 4, 4 (2013), 19.

Digital Library

[17]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).

[18]

William R Miller and Stephen Rollnick. 2012. Motivational interviewing: Helping people change. Guilford press.

[19]

Peter M Monti, Nadine R Mastroleo, Nancy P Barnett, Suzanne M Colby, Christopher W Kahler, and Don Operario. 2016. Brief motivational intervention to reduce alcohol and HIV/sexual risk behavior in emergency department patients: A randomized controlled trial. Journal of consulting and clinical psychology, Vol. 84, 7 (2016), 580.

[20]

Theresa Moyers, Tim Martin, Delwyn Catley, Kari Jo Harris, and Jasjit S Ahluwalia. 2003. Assessing the integrity of motivational interviewing interventions: Reliability of the motivational interviewing skills code. Behavioural and Cognitive Psychotherapy, Vol. 31, 2 (2003), 177--184.

[21]

United Nations Office on Drugs and Crime. 2019. World Drug Report 2019. 387 pages. https://doi.org/https://doi.org/10.18356/a4dd519a-en

[22]

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic Differentiation in PyTorch. In NIPS Autodiff Workshop.

[23]

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, Vol. 12 (2011), 2825--2830.

Digital Library

[24]

Verónica Pérez-Rosas, Rada Mihalcea, Kenneth Resnicow, Satinder Singh, Lawrence Ann, Kathy J Goggin, and Delwyn Catley. 2017. Predicting counselor behaviors in motivational interviewing encounters. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. 1128--1137.

[25]

Radim v Rehr uv rek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. ELRA, Valletta, Malta, 45--50. http://is.muni.cz/publication/884893/en.

[26]

Sune Rubak, Annelli Sandbæk, Torsten Lauritzen, and Bo Christensen. 2005. Motivational interviewing: a systematic review and meta-analysis. Br J Gen Pract, Vol. 55, 513 (2005), 305--312.

[27]

Daniel Schulman, Timothy Bickmore, and Candace Sidner. 2011. An intelligent conversational agent for promoting long-term health behavior change using motivational interviewing. In 2011 AAAI Spring Symposium Series.

[28]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.

[29]

Byron C Wallace, Issa J Dahabreh, Thomas A Trikalinos, Michael Barton Laws, Ira Wilson, and Eugene Charniak. 2014a. Identifying differences in physician communication styles with a log-linear transition component model. In Twenty-Eighth AAAI Conference on Artificial Intelligence.

Digital Library

[30]

Byron C Wallace, M Barton Laws, Kevin Small, Ira B Wilson, and Thomas A Trikalinos. 2014b. Automatically annotating topics in transcripts of patient-provider interactions via machine learning. Medical Decision Making, Vol. 34, 4 (2014), 503--512.

[31]

Byron C Wallace, Thomas A Trikalinos, M Barton Laws, Ira B Wilson, and Eugene Charniak. 2013. A generative joint, additive, sequential model of topics and speech acts in patient-doctor communication. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 1765--1775.

[32]

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R'emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. HuggingFace's Transformers: State-of-the-art Natural Language Processing. ArXiv, Vol. abs/1910.03771 (2019).

[33]

Ugan Yasavur, Christine Lisetti, and Naphtali Rishe. 2014. Intelligent virtual agents and spoken dialog systems come together to deliver brief health interventions. Journal on Multimodal User Interfaces, in press, Vol. 1, 1 (2014), 19.

[34]

Steve Young, Milica Gavs ić, Blaise Thomson, and Jason D Williams. 2013. Pomdp-based statistical spoken dialog systems: A review. Proc. IEEE, Vol. 101, 5 (2013), 1160--1179.

[35]

S Zhou, T Bickmore, A Rubin, C Yeksigian, R Lippin-Foster, M Heilman, and SR Simon. 2017. A relational agent for alcohol misuse screening and intervention in primary care. In CHI 2017 Workshop on Interactive Systems in Healthcare (WISH).

Cited By

Steenstra IMurali PPerkins RJoseph NPaasche-Orlow MBickmore T(2024)Engaging and Entertaining Adolescents in Health Education Using LLM-Generated Fantasy Narrative Games and Virtual AgentsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650983(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650983
de Souza PSantos VAlves BProença FMotti VRodrigues KNeris V(2024)Case Study: End users in recovery from substance use disorders as designers and developers of digital games with therapeutic potentialExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3637114(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3637114
Phelan CHeyer JPfafman RKerrigan CTzilos Wernette GDombrowski LMiller APater J(2022)The Work of Digital Social Re-entry in Substance Use Disorder RecoveryProceedings of the ACM on Human-Computer Interaction10.1145/35556586:CSCW2(1-33)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555658
Show More Cited By

Index Terms

Towards a Computational Framework for Automating Substance Use Counseling with Virtual Agents

Recommendations

Motivating Health Behavior Change with Humorous Virtual Agents
IVA '20: Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents

Research on the psychology of humor indicates that humor can increase the impact of persuasive messages in certain circumstances, suggesting that virtual agents could use humor to improve the effectiveness of motivational counseling with users. To ...
Coerced Change-talk with Conversational Agents Promotes Confidence in Behavior Change
PervasiveHealth'19: Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare

Motivational interviewing is a counseling technique that works, in part, by helping people talk about changing their behavior. We describe two conversational agent-based interventions to increase motivation and confidence to promote physical activity ...
Exploring Virtual Agents for Augmented Reality
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Prior work has shown that embodiment can benefit virtual agents, such as increasing rapport and conveying non-verbal information. However, it is unclear if users prefer an embodied to a speech-only agent for augmented reality (AR) headsets that are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

AAMAS '20: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems

May 2020

2289 pages

ISBN:9781450375184

General Chairs:
Amal El Fallah Seghrouchni
Sorbonne University, France
,
Gita Sukthankar
University of Central Florida, United States
,
Program Chairs:
Bo An
Nanyang Technological University, Singapore
,
Neil Yorke-Smith Yorke-Smith
Delft University of Technology, Netherlands

Sponsors

Publisher

International Foundation for Autonomous Agents and Multiagent Systems

Richland, SC

Publication History

Published: 13 May 2020

Check for updates

Author Tags

Qualifiers

Research-article

Conference

AAMAS '19

Sponsor:

SIGAI

AAMAS '19: International Conference on Autonomous Agents and Multiagent Systems

May 9 - 13, 2020

Auckland, New Zealand

Acceptance Rates

Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
101
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Steenstra IMurali PPerkins RJoseph NPaasche-Orlow MBickmore T(2024)Engaging and Entertaining Adolescents in Health Education Using LLM-Generated Fantasy Narrative Games and Virtual AgentsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650983(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650983
de Souza PSantos VAlves BProença FMotti VRodrigues KNeris V(2024)Case Study: End users in recovery from substance use disorders as designers and developers of digital games with therapeutic potentialExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3637114(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3637114
Phelan CHeyer JPfafman RKerrigan CTzilos Wernette GDombrowski LMiller APater J(2022)The Work of Digital Social Re-entry in Substance Use Disorder RecoveryProceedings of the ACM on Human-Computer Interaction10.1145/35556586:CSCW2(1-33)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555658
Pater JPhelan CCornet VAhmed RColletta SHess EKerrigan CToscos T(2021)User-Centered Design of a Mobile App to Support Peer Recovery in a Clinical SettingProceedings of the ACM on Human-Computer Interaction10.1145/34491865:CSCW1(1-31)Online publication date: 22-Apr-2021
https://dl.acm.org/doi/10.1145/3449186

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents