Nothing Special   »   [go: up one dir, main page]

Skip to main content
Log in

Informing the design of question-asking conversational agents for reflection

  • Original Paper
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Reflecting on everyday experiences offers valuable insights and has the potential to enhance psychological well-being. Yet, only some have access to a facilitator for reflection. Conversational agents hold promise as companions for these discussions. We surveyed individuals with therapy experience to understand user needs and arrived at interaction strategies used in therapy. We then evaluated these strategies with five therapists and transformed our data, along with their input, into a set of interaction strategies to be used on conversational agents for reflection. We developed an AI chatbot prototype where we implemented these strategies and conducted a 1-week in-the-wild study with 34 participants to evaluate the interaction strategies and experiences of interacting with a chatbot for reflection. Findings reveal that participants are willing to engage with a chatbot, even with limited capabilities. Critical aspects include the chatbot’s contextual awareness, statement repetition, and human-like qualities. Successfully balancing questions with non-question statements is essential for a pleasurable dialogue-driven reflection. Our paper presents implications for future design and research studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

Data availability

The data supporting this study’s findings are available on Koç University Servers and have restricted access. The data will be available upon request.

Notes

  1. https://www.qualtrics.com/

  2. https://miro.com/

  3. https://openai.com/

  4. https://www.voiceflow.com/

  5. https://telegram.org/

  6. https://www.taguette.org/

References

  1. Staudinger UM (2001) Life reflection: a social–cognitive analysis of life review. Rev Gen Psychol 5:148–160. https://doi.org/10.1037/1089-2680.5.2.148

    Article  Google Scholar 

  2. Baumer EPS, Khovanskaya V, Matthews M, et al (2014) Reviewing reflection: on the use of reflection in interactive system design. In: Proceedings of the 2014 Conference on Designing Interactive Systems. ACM, New York, NY, USA, pp 93–102

  3. Mezirow J (1991) Transformative dimensions of adult learning. Jossey-Bass

    Google Scholar 

  4. Stein D, Grant AM (2014) Disentangling the relationships among self-reflection, insight, and subjective well-being: the role of dysfunctional attitudes and core self-evaluations. J Psychol 148:505–522. https://doi.org/10.1080/00223980.2013.810128

    Article  Google Scholar 

  5. Fleck R, Fitzpatrick G (2010) Reflecting on reflection. In: Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction - OZCHI ’10. ACM Press, New York, New York, USA, p 216

  6. Gelter H (2003) Why is reflective thinking uncommon. Reflective Pract 4:337–344. https://doi.org/10.1080/1462394032000112237

    Article  Google Scholar 

  7. Li I, Dey A, Forlizzi J (2010) A stage-based model of personal informatics systems. In: Proceedings of the 28th international conference on Human factors in computing systems - CHI ’10. ACM Press, New York, New York, USA, p 557

  8. Fan X, Luo W, Menekse M, et al (2017) Scaling reflection prompts in large classrooms via mobile interfaces and natural language processing. In: Proceedings of the 22Nd International Conference on Intelligent User Interfaces. ACM, New York, NY, USA, pp 363–374

  9. Isaacs E, Konrad A, Walendowski A, et al (2013) Echoes from the past: how technology mediated reflection improves well-being. Conference on Human Factors in Computing Systems - Proceedings 1071–1080. https://doi.org/10.1145/2470654.2466137

  10. Bentvelzen M, Woźniak PW, Herbes PSF et al (2022) Revisiting reflection in HCI. Proc ACM Interact Mob Wearable Ubiquitous Technol 6:1–27. https://doi.org/10.1145/3517233

    Article  Google Scholar 

  11. Mols I, van den Hoven E, Eggen B (2016) Technologies for everyday life reflection: illustrating a design space. In: Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. ACM, New York, NY, USA, pp 53–61

  12. Scholten MR, Kelders SM, Van Gemert-Pijnen JE (2017) Self-guided web-based interventions: scoping review on user needs and the potential of embodied conversational agents to address them. J Med Internet Res 19:e383. https://doi.org/10.2196/jmir.7351

    Article  Google Scholar 

  13. Ho A, Hancock J, Miner AS (2018) Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. J Commun 68:712–733. https://doi.org/10.1093/joc/jqy026

    Article  Google Scholar 

  14. Park H, Lee J (2021) Designing a conversational agent for sexual assault survivors: defining burden of self-disclosure and envisioning survivor-centered solutions. Conference on Human Factors in Computing Systems - Proceedings 17. https://doi.org/10.1145/3411764.3445133

  15. Zhang Y, Parker AG Eat4Thought: a design of food journaling. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3334480

  16. Sun Z, Wang S, Yang W, et al (2020) A postcard from your food journey in the past": Promoting self-reflection on social food posting. DIS 2020 - Proceedings of the 2020 ACM Designing Interactive Systems Conference 1819–1832. https://doi.org/10.1145/3357236.3395475

  17. Luo Y, Kim Y-H, Lee B et al (2021) FoodScrap: promoting rich data capture and reflective food journaling through speech input. Designing Interactive Systems Conference 2021. ACM, New York, NY, USA, pp 606–618

    Chapter  Google Scholar 

  18. Jörke M, Sefidgar YS, Massachi T, et al (2023) Pearl: a technology probe for machine-assisted reflection on personal data. In: Proceedings of the 28th International Conference on Intelligent User Interfaces. ACM, New York, NY, USA, pp 902–918

  19. Kocielnik R, Avrahami D, Marlow J, et al (2018) Designing for workplace reflection: a chat and voice-based conversational agent. DIS 2018 - Proceedings of the 2018 Designing Interactive Systems Conference 881–894. https://doi.org/10.1145/3196709.3196784

  20. Guillou H, Chow K, Fritz T, McGrenere J (2020) Is your time well spent? Reflecting on knowledge work more holistically. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp 1–9

  21. Schön DA (2017) The reflective practitioner: how professionals think in action. The Reflective Practitioner: How Professionals Think in Action 1–374. https://doi.org/10.4324/9781315237473/REFLECTIVE-PRACTITIONER-DONALD-SCH

  22. McCarthy J, Wright P (2004) Technology as experience. https://doi.org/10.7551/mitpress/6687.001.0001

  23. Sengers P, Boehner K, David S, Kaye J (2005) Reflective design. Critical computing—between Sense and Sensibility - Proceedings of the 4th Decennial Aarhus Conference 49–58. https://doi.org/10.1145/1094562.1094569

  24. Al-quaran M, Geißler D, Kamoen C et al (2022) MoodTurner: A self-tracking smart jewellery to support awareness and reflection in sensory processing sensitivity self-care. CHI Conference on Human Factors in Computing Systems Extended Abstracts. ACM, New York, NY, USA, pp 1–6

    Google Scholar 

  25. Thieme A, Comber R, Miebach J, et al (2012) “We’ve bin watching you”-Designing for reflection and social persuasion to promote sustainable lifestyles. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2207676

  26. Ganglbauer E, Fitzpatrick G, Güldenpfennig F (2015) Why and what did we throw out?: Probing on reflection through the food waste diary. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp 1105–1114

  27. Mols I, van den Hoven E, Eggen B (2017) Balance, Cogito and Dott: exploring media modalities for everyday-life reflection. In: Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction. ACM, New York, NY, USA, pp 427–433

  28. Mols I, van den Hoven E, Eggen B (2016) Informing design for reflection. In: Proceedings of the 9th Nordic Conference on Human-Computer Interaction. ACM, New York, NY, USA, pp 1–10

  29. Baumer EPS (2015) Reflective informatics: conceptual dimensions for designing technologies of reflection. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp 585–594

  30. Mols I, van den Hoven E, Eggen B (2020) Everyday life reflection. In: Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction. ACM, New York, NY, USA, pp 67–79

  31. Karaturhan P, Arıkan E, Durak P et al (2022) Combining momentary and retrospective self-reflection in a mobile photo-based journaling application. Nordic Human-Computer Interaction Conference. ACM, New York, NY, USA, pp 1–12

    Google Scholar 

  32. Flobak E, Nordberg OE, Guribye F et al (2021) “This is the story of me”: designing audiovisual narratives to support reflection on cancer journeys. Designing Interactive Systems Conference 2021. ACM, New York, NY, USA, pp 1031–1045

    Chapter  Google Scholar 

  33. Wagener N, Reicherts L, Zargham N, et al (2023) SelVReflect: A guided VR experience fostering reflection on personal challenges. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp 1–17

  34. Peng F, Labelle VC, Picard RW, Yue EC (2018) A trip to the moon: Personalized animated movies for self-reflection. Conference on Human Factors in Computing Systems - Proceedings 2018-April:1–10. https://doi.org/10.1145/3173574.3173827

  35. Papachristos E, Meldgaard DP, Thomsen IR, Skov MB (2021) ReflectPal: exploring self-reflection on collaborative activities using voice assistants. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12935 LNCS:187–208. https://doi.org/10.1007/978-3-030-85610-6_12

  36. Kocielnik R, Xiao L, Avrahami D, Hsieh G (2018) Reflection companion: a conversational system for engaging users in reflection on physical activity. Proc ACM Interact Mob Wearable Ubiquitous Technol 2:1–26. https://doi.org/10.1145/3214273

    Article  Google Scholar 

  37. Karyda M, Mekler ED, Lucero A (2021) Data agents: promoting reflection through meaningful representations of personal data in everyday life. Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3411764.3445112

  38. Thudt A, Hinrichs U, Huron S, Carpendale S (2018) Self-reflection and personal physicalization construction. Conference on Human Factors in Computing Systems - Proceedings 2018-April: https://doi.org/10.1145/3173574.3173728

  39. Hughes HW, Kooy M, Kanevsky L (1997) Dialogic reflection and journaling. Clear House: J Educ Strateg, Issues Ideas 70:187–190. https://doi.org/10.1080/00098655.1997.10544193

    Article  Google Scholar 

  40. Terzimehić N, Schött SY, Bemmann F, Buschek D (2021) MEMEories: Internet Memes as Means for Daily Journaling. DIS 2021 - Proceedings of the 2021 ACM Designing Interactive Systems Conference: Nowhere and Everywhere 2021:538–548. https://doi.org/10.1145/3461778.3462080

  41. Ryou J, Lee K, Lee J (2023) “You said da da...”: A short echoing tweak for journaling with VA. In: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp 1–6

  42. Rapp A, Tirassa M (2017) Know thyself: a theory of the self for personal informatics. Hum Comput Interact 32:335–380. https://doi.org/10.1080/07370024.2017.1285704

    Article  Google Scholar 

  43. Hayes SC, Hofmann SG (2017) The third wave of cognitive behavioral therapy and the rise of process-based care. World Psychiatry 16:245–246. https://doi.org/10.1002/wps.20442

    Article  Google Scholar 

  44. Campbell D, Draper R (1984) Applications of systemic family therapy circular interviewing : a multifaceted edited by

  45. Linley PA, Joseph S (2004) Positive psychology in practice. Wiley

    Book  Google Scholar 

  46. Parloff MB, Kelman HC, Frank JD (1954) Comfort, effectiveness, and self-awareness as criteria of improvement in psychotherapy. Am J Psychiatry 111:343–352. https://doi.org/10.1176/ajp.111.5.343

    Article  Google Scholar 

  47. Carey TA, Mullan RJ (2004) What is socratic questioning? Psychotherapy: Theory. Research, Practice, Training 41:217–226. https://doi.org/10.1037/0033-3204.41.3.217

    Article  Google Scholar 

  48. Brown J (1997) Circular questioning: an introductory guide. Aust N Z J Fam Ther 18:109–114. https://doi.org/10.1002/j.1467-8438.1997.tb00276.x

    Article  Google Scholar 

  49. Tomm K (1987) Interventive Interviewing: Part II. Reflexive questioning as a means to enable self-healing. Fam Process 26:167–183. https://doi.org/10.1111/j.1545-5300.1987.00167.x

    Article  Google Scholar 

  50. Williams EN (2023) The use of questions in psychotherapy: a review of research on immediate outcomes. Psychotherapy 60:246–254. https://doi.org/10.1037/pst0000471

    Article  Google Scholar 

  51. Weizenbaum J (1966) ELIZA—a computer program for the study of natural language communication between man and machine. Commun ACM 9:36–45. https://doi.org/10.1145/365153.365168

    Article  MathSciNet  Google Scholar 

  52. Fitzpatrick KK, Darcy A, Vierhile M (2017) Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Ment Health 4:e19. https://doi.org/10.2196/mental.7785

    Article  Google Scholar 

  53. Tudor Car L, Dhinagaran DA, Kyaw BM et al (2020) Conversational agents in health care: scoping review and conceptual analysis. J Med Internet Res 22:e17158. https://doi.org/10.2196/17158

    Article  Google Scholar 

  54. Lee M, Ackermans S, Van As N, et al (2019) Caring for Vincent: a chatbot for self-compassion. Conference on Human Factors in Computing Systems - Proceedings 1–13. https://doi.org/10.1145/3290605.3300932

  55. Narain J, Quach T, Davey M et al (2020) Promoting wellbeing with Sunny, a chatbot that facilitates positive messages within social groups. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp 1–8

    Google Scholar 

  56. Falala-Sechet C, Antoine L, Thiriez I, Bungener C (2019) Owlie: A chatbot that provides emotional support for coping with psychological difficulties. IVA 2019 - Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents 236–237. https://doi.org/10.1145/3308532.3329416

  57. Ly KH, Ly AM, Andersson G (2017) A fully automated conversational agent for promoting mental well-being: A pilot RCT using mixed methods. Internet Interv 10:39–46. https://doi.org/10.1016/J.INVENT.2017.10.002

    Article  Google Scholar 

  58. Park S, Thieme A, Han J, et al (2021) “I wrote as if I were telling a story to someone I knew.”: Designing Chatbot Interactions for Expressive Writing in Mental Health. In: Designing Interactive Systems Conference 2021. ACM, New York, NY, USA, pp 926–941

  59. González AL, Young JE (2020) Please Tell Me About It. In: Proceedings of the 8th International Conference on Human-Agent Interaction. ACM, New York, NY, USA, pp 266–268

  60. Chen Z, Lu Y, Nieminen MP, Lucero A (2020) Creating a chatbot for and with migrants. In: Proceedings of the 2020 ACM Designing Interactive Systems Conference. ACM, New York, NY, USA, pp 219–230

  61. Garg R, Sengupta S (2020) Conversational technologies for in-home learning: using co-design to understand children’s and parents’ perspectives. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp 1–13

  62. Piccolo LSG, Troullinou P, Alani H (2021) Chatbots to support children in coping with online threats: socio-technical requirements. Designing Interactive Systems Conference 2021. ACM, New York, NY, USA, pp 1504–1517

    Chapter  Google Scholar 

  63. Kim J, Kim Y, Kim B et al (2018) Can a machine tend to teenagers’ emotional needs? extended abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp 1–6

    Google Scholar 

  64. Bordin ES (1979) The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory. Research & Practice 16:252–260. https://doi.org/10.1037/h0085885

    Article  Google Scholar 

  65. Elliott R (2008) Research on client experiences of therapy: introduction to the special section. Psychother Res 18:239–242. https://doi.org/10.1080/10503300802074513

    Article  Google Scholar 

  66. Elliott R, James E (1989) Varieties of client experience in psychotherapy: an analysis of the literature. Clin Psychol Rev 9:443–467. https://doi.org/10.1016/0272-7358(89)90003-2

    Article  Google Scholar 

  67. Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3:77–101. https://doi.org/10.1191/1478088706qp063oa

    Article  Google Scholar 

  68. Geldard D, Geldard K, Yin Foo R (2017) Basic personal counselling: a training manual for counsellors. Cengage AU

  69. Hill CE, Knox S, Duan C (2023) Psychotherapist advice, suggestions, recommendations: a research review. Psychotherapy 60:295–305. https://doi.org/10.1037/pst0000476

    Article  Google Scholar 

  70. Larsen D, Edey W, Lemay L (2007) Understanding the role of hope in counselling: exploring the intentional uses of hope. Couns Psychol Q 20:401–416. https://doi.org/10.1080/09515070701690036

    Article  Google Scholar 

  71. Hauk N, Hüffmeier J, Krumm S (2018) Ready to be a silver surfer? A meta-analysis on the relationship between chronological age and technology acceptance. Comput Human Behav 84:304–319. https://doi.org/10.1016/j.chb.2018.01.020

    Article  Google Scholar 

  72. Grant AM (2001) Rethinking Psychological mindedness: metacognition, self-reflection, and insight. Behav Chang 18:8–17. https://doi.org/10.1375/bech.18.1.8

    Article  Google Scholar 

  73. Karaturhan P, Yantaç AE, Kuscu K (2021) Layer zero: an approach for deepening self-reflection on instagram shares. Springer International Publishing, Cham, pp 352–372

    Google Scholar 

  74. Ukrop M, Švábenský V, Nehyba J (2019) Reflective diary for professional development of novice teachers. In: Proceedings of the 50th ACM Technical Symposium on Computer Science Education. ACM, New York, NY, USA, pp 1088–1094

  75. Xu A, Liu Z, Guo Y, et al (2017) A new chatbot for customer service on social media. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp 3506–3510

  76. Morris RR, Kouddous K, Kshirsagar R, Schueller SM (2018) Towards an artificially empathic conversational agent for mental health applications: system design and user perceptions. J Med Internet Res 20:e10148. https://doi.org/10.2196/10148

    Article  Google Scholar 

  77. de Souza PM, Pires I da C, Motti VG, et al (2022) Design recommendations for chatbots to support people with depression. In: Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems. ACM, New York, NY, USA, pp 1–11

  78. Daher K, Casas J, Khaled OA, Mugellini E (2020) Empathic chatbot response for medical assistance. In: Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents. ACM, New York, NY, USA, pp 1–3

  79. De Nieva JO, Joaquin JA, Tan CB, et al (2020) Investigating students use of a mental health chatbot to alleviate academic stress. ACM International Conference Proceeding Series 1–10. https://doi.org/10.1145/3431656.3431657

  80. Nass C, Steuer J, Tauber ER (1994) Computers are social actors. In: Proceedings of the SIGCHI conference on Human factors in computing systems celebrating interdependence - CHI ’94. ACM Press, New York, New York, USA, pp 72–78

  81. Nass C, Moon Y (2000) Machines and mindlessness: social responses to computers. J Soc Issues 56:81–103. https://doi.org/10.1111/0022-4537.00153

    Article  Google Scholar 

  82. Gnewuch U, Morana S, Adam M, Maedche A (2018) Faster is not always better: understanding the effect of dynamic response delays in human-chatbot interaction

  83. Cho E, Motalebi N, Sundar SS, Abdullah S (2022) Alexa as an active listener: how backchanneling can elicit self-disclosure and promote user experience. Proc ACM Hum Comput Interact 6:1–23. https://doi.org/10.1145/3555164

    Article  Google Scholar 

  84. Maeng W, Lee J (2021) Designing a chatbot for survivors of sexual violence: exploratory study for hybrid approach combining rule-based chatbot and ML-based chatbot. Asian CHI Symposium 2021. ACM, New York, NY, USA, pp 160–166

    Chapter  Google Scholar 

  85. Huang M, Zhu X, Gao J (2020) Challenges in building intelligent open-domain dialog systems. ACM Trans Inf Syst 38:1–32. https://doi.org/10.1145/3383123

    Article  Google Scholar 

  86. Fleuridas C, Nelson TS, Rosenthal DM (1986) The evolution of circular questions: training family therapists. J Marital Fam Ther 12:113–127. https://doi.org/10.1111/j.1752-0606.1986.tb01629.x

    Article  Google Scholar 

  87. Padesky CA (1993) Padesky, Christine A. "Socratic questioning: changing minds or guiding discovery. In: A keynote address delivered at the European Congress of Behavioural and Cognitive Therapies, London. London

  88. Liu C, Jiang J, Xiong C, et al (2020) Towards building an intelligent chatbot for customer service. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, New York, NY, USA, pp 3377–3385

  89. You Y, Tsai C-H, Li Y et al (2023) Beyond Self-diagnosis: how a chatbot-based symptom checker should respond. ACM Transactions on Computer-Human Interaction 30:1–44. https://doi.org/10.1145/3589959

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the study’s conception and design. Pelin Karaturhan performed material preparation and data collection. Pelin Karaturhan and İlayda Orhan performed data analysis. Pelin Karaturhan wrote the first draft of the manuscript, and all authors commented on previous versions. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Pelin Karaturhan.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karaturhan, P., Orhan, İ., Kuşcu, K. et al. Informing the design of question-asking conversational agents for reflection. Pers Ubiquit Comput (2024). https://doi.org/10.1007/s00779-024-01831-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00779-024-01831-7

Keywords

Navigation