extended-abstract

Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational Search

Authors:

Sachin Pathiyan Cherumanal,

Johanne R. Trippas,

Danula Hettiachchi,

Flora D. Salim,

Damiano SpinaAuthors Info & Claims

MobileHCI '24 Adjunct: Adjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction

Article No.: 10, Pages 1 - 10

https://doi.org/10.1145/3640471.3680245

Published: 21 September 2024 Publication History

Abstract

Spoken Conversational Search (SCS) poses unique challenges in understanding user-system interactions due to the absence of visual cues, and the complexity of less structured dialogue. Tackling the impacts of cognitive bias in today’s information-rich online environment, especially when SCS becomes more prevalent, this paper integrates insights from information science, psychology, cognitive science, and wearable sensor technology to explore potential opportunities and challenges in studying cognitive biases in SCS. It then outlines a framework for experimental designs with various experiment setups to multimodal instruments. It also analyzes data from an existing dataset as a preliminary example to demonstrate the potential of this framework and discuss its implications for future research. In the end, it discusses the challenges and ethical considerations associated with implementing this approach. This work aims to provoke new directions and discussion in the community and enhance understanding of cognitive biases in Spoken Conversational Search.

Supplemental Material

MP4 File

Teaser Video

Download
80.41 MB

References

[1]

Evan Ackerman and Eliza Strickland. 2022. Are You Ready for Workplace Brain Scanning?: Leveraging brain data will make workers happier and more productive, backers say. IEEE Spectrum 59, 12 (2022), 46–52. https://doi.org/10.1109/MSPEC.2022.9976479

[2]

Yamen Ajjour, Henning Wachsmuth, Johannes Kiesel, Martin Potthast, Matthias Hagen, and Benno Stein. 2019. Data Acquisition for Argument Search: The args.me corpus. In 42nd German Conference on Artificial Intelligence (KI 2019), Christoph Benzmüller and Heiner Stuckenschmidt (Eds.). Springer, Berlin Heidelberg New York, 48–59. https://doi.org/10.1007/978-3-030-30179-8_4

Digital Library

[3]

Mohammad Aliannejadi and Johanne R. Trippas. 2022. Conversational Information Seeking: Theory and Evaluation: CHIIR 2022 Half Day Tutorial. In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval (Regensburg, Germany) (CHIIR ’22). Association for Computing Machinery, New York, NY, USA, 365–366. https://doi.org/10.1145/3498366.3505843

Digital Library

[4]

Marco Allegretti, Yashar Moshfeghi, Maria Hadjigeorgieva, Frank E. Pollick, Joemon M. Jose, and Gabriella Pasi. 2015. When Relevance Judgement is Happening? An EEG-Based Study. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (Santiago, Chile) (SIGIR ’15). Association for Computing Machinery, New York, NY, USA, 719–722. https://doi.org/10.1145/2766462.2767811

Digital Library

[5]

Pablo Arnau-González, Stamos Katsigiannis, Miguel Arevalillo-Herráez, and Naeem Ramzan. 2021. BED: A New Data Set for EEG-based Biometrics. IEEE Internet of Things Journal 8, 15 (2021), 12219–12230. https://doi.org/10.1109/JIOT.2021.3061727

[6]

Resham Arya, Jaiteg Singh, and Ashok Kumar. 2021. A survey of multidisciplinary domains contributing to affective computing. Computer Science Review 40 (2021), 100399. https://doi.org/10.1016/j.cosrev.2021.100399

Digital Library

[7]

Erdrin Azemi, Ali Moin, Anuranjini Pragada, Jean Hsiang-Chun Lu, Victoria M Powell, Juri Minxha, and Steven P Hotelling. 2023. Biosignal sensing device using dynamic selection of electrodes. US Patent App. 18/094,841.

[8]

Leif Azzopardi. 2021. Cognitive Biases in Search: A Review and Reflection of Cognitive Biases in Information Retrieval. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval (Canberra ACT, Australia) (CHIIR ’21). Association for Computing Machinery, New York, NY, USA, 27–37. https://doi.org/10.1145/3406522.3446023

Digital Library

[9]

Ebrahim Babaei, Benjamin Tag, Tilman Dingler, and Eduardo Velloso. 2021. A Critique of Electrodermal Activity Practices at CHI. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 177, 14 pages. https://doi.org/10.1145/3411764.3445370

Digital Library

[10]

Bence Bago, Darren Frey, Julie Vidal, Olivier Houdé, Gregoire Borst, and Wim De Neys. 2018. Fast and slow thinking: Electrophysiological evidence for early conflict sensitivity. Neuropsychologia 117 (2018), 483–490. https://doi.org/10.1016/j.neuropsychologia.2018.07.017

[11]

Garbiel Bénédict, Ruqing Zhang, and Donald Metzler. 2023. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (Taipei, Taiwan) (SIGIR ’23). Association for Computing Machinery, New York, NY, USA, 3460–3463. https://doi.org/10.1145/3539618.3591923

Digital Library

[12]

Amin Bigdeli, Negar Arabzadeh, Shirin Seyedsalehi, Morteza Zihayat, and Ebrahim Bagheri. 2023. Understanding and Mitigating Gender Bias in Information Retrieval Systems. In European Conference on Information Retrieval. Springer, 315–323. https://doi.org/10.1007/978-3-031-28241-6_32

Digital Library

[13]

Nattapat Boonprakong, Xiuge Chen, Catherine Davey, Benjamin Tag, and Tilman Dingler. 2023. Bias-Aware Systems: Exploring Indicators for the Occurrences of Cognitive Biases When Facing Different Opinions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 27, 19 pages. https://doi.org/10.1145/3544548.3580917

Digital Library

[14]

Nattapat Boonprakong, Xiuge Chen, Catherine Davey, Benjamin Tag, and Tilman Dingler. 2023. Bias-Aware Systems: Exploring Indicators for the Occurrences of Cognitive Biases When Facing Different Opinions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 27, 19 pages. https://doi.org/10.1145/3544548.3580917

Digital Library

[15]

Patricia J. Bota, Chen Wang, Ana L.N. Fred, and Hugo Placido Da Silva. 2019. A Review, Current Challenges, and Future Possibilities on Emotion Recognition Using Machine Learning and Physiological Signals. IEEE Access 7 (2019), 140990–141020. https://doi.org/10.1109/ACCESS.2019.2944001

[16]

Danielle Bragg, Cynthia Bennett, Katharina Reinecke, and Richard Ladner. 2018. A Large Inclusive Study of Human Listening Rates. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3174018

Digital Library

[17]

Georg Buscher, Andreas Dengel, Ralf Biedert, and Ludger V. Elst. 2012. Attentive documents: Eye tracking as implicit feedback for information retrieval and beyond. ACM Trans. Interact. Intell. Syst. 1, 2, Article 9 (jan 2012), 30 pages. https://doi.org/10.1145/2070719.2070722

Digital Library

[18]

Donald O Case, James E Andrews, J David Johnson, and Suzanne L Allard. 2005. Avoiding versus seeking: the relationship of information seeking to avoidance, blunting, coping, dissonance, and related concepts. Journal of the Medical Library Association 93, 3 (2005), 353.

[19]

Sachin Pathiyan Cherumanal, Ujwal Gadiraju, and Damiano Spina. 2024. Everything We Hear: Towards Tackling Misinformation in Podcasts. arXiv preprint arXiv:2408.00292 (2024). https://arxiv.org/pdf/2408.00292

[20]

Aleksandr Chuklin, Aliaksei Severyn, Johanne R. Trippas, Enrique Alfonseca, Hanna Silen, and Damiano Spina. 2019. Using Audio Transformations to Improve Comprehension in Voice Question Answering. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings (Lugano, Switzerland). Springer-Verlag, Berlin, Heidelberg, 164–170. https://doi.org/10.1007/978-3-030-28577-7_12

Digital Library

[21]

Russ Clay, Jessica M. Barber, and Natalie J. Shook. 2013. Techniques for Measuring Selective Exposure: A Critical Review. Communication Methods and Measures 7, 3-4 (2013), 147–171. https://doi.org/10.1080/19312458.2013.813925

[22]

Michael J. Cole, Chathra Hendahewa, Nicholas J. Belkin, and Chirag Shah. 2014. Discrimination between Tasks with User Activity Patterns during Information Search. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval (Gold Coast, Queensland, Australia) (SIGIR ’14). Association for Computing Machinery, New York, NY, USA, 567–576. https://doi.org/10.1145/2600428.2609591

Digital Library

[23]

Fabio Crestani and Heather Du. 2006. Written versus spoken queries: A qualitative and quantitative comparative analysis. Journal of the American Society for Information Science and Technology 57, 7 (2006), 881–890. https://doi.org/10.1002/asi.20350

[24]

Kahneman Daniel. 2017. Thinking, Fast and Slow. Farrar, Straus and Giroux.

[25]

Tilman Dingler, Benjamin Tag, David A. Eccles, Niels van Berkel, and Vassilis Kostakos. 2022. Method for Appropriating the Brief Implicit Association Test to Elicit Biases in Users. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 243, 16 pages. https://doi.org/10.1145/3491102.3517570

Digital Library

[26]

Tim Draws, Nava Tintarev, Ujwal Gadiraju, Alessandro Bozzon, and Benjamin Timmermans. 2021. This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Canada) (SIGIR ’21). Association for Computing Machinery, New York, NY, USA, 295–305. https://doi.org/10.1145/3404835.3462851

Digital Library

[27]

Mateusz Dubiel, Martin Halvey, Leif Azzopardi, and Sylvain Daronnat. 2018. Investigating how conversational search agents affect user’s behaviour, performance and search experience. In The second international workshop on conversational approaches to information retrieval. https://doi.org/10.1037/0022-3514.51.1.55

[28]

Roger A Elkin and Michael R Leippe. 1986. Physiological arousal, dissonance, and attitude change: evidence for a dissonance-arousal link and a" don’t remind me" effect.Journal of personality and social psychology 51, 1 (1986), 55.

[29]

Emotiv. 2023. MN8 - 2 Channel EEG Earbuds. https://www.emotiv.com/mn8/ Accessed: October 21, 2023.

[30]

Robert Epstein and Ronald E Robertson. 2015. The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proceedings of the National Academy of Sciences 112, 33 (2015), E4512–E4521. https://doi.org/10.1073/pnas.1419828112

[31]

Manuel JA Eugster, Tuukka Ruotsalo, Michiel M Spapé, Oswald Barral, Niklas Ravaja, Giulio Jacucci, and Samuel Kaski. 2016. Natural brain-information interfaces: Recommending information by relevance inferred from human brain signals. Scientific reports 6, 1 (2016), 38580. https://doi.org/10.1038/srep38580

[32]

Kara D Federmeier, Suzanne R Jongman, and Jakub M Szewczyk. 2020. Examining the role of general cognitive skills in language processing: a window into complex cognition. Current directions in psychological science 29, 6 (2020), 575–582. https://doi.org/10.1177/0963721420964095

[33]

Christos Fidas, Stella Sylaiou, Tsvi Kuflik, Fotis Liarokapis, Panayiotis Koutsabasis, Manolis Wallace, Dimitrios Koukopoulos, Angeliki Antoniou, Katerina Mania, and Ioanna Lykourentzou. 2023. Mobile and Multimodal HCI Design Approaches in Museums for People with Impairments. In Proceedings of the 25th International Conference on Mobile Human-Computer Interaction (Athens, Greece) (MobileHCI ’23 Companion). Association for Computing Machinery, New York, NY, USA, Article 33, 4 pages. https://doi.org/10.1145/3565066.3609509

Digital Library

[34]

Alexander L Francis and Jordan Love. 2020. Listening effort: Are we measuring cognition or affect, or both?Wiley Interdisciplinary Reviews: Cognitive Science 11, 1 (2020), e1514.

[35]

Alexander Frummet, Andrea Papenmeier, Maik Fröbe, Johannes Kiesel, Vaibhav Adlakha, Norbert Braunschweiler, Mateusz Dubiel, Satanu Ghosh, Marcel Gohsen, Christin Kreutz, Milad Momeni, Markus Nilles, Sachin Pathiyan Cherumanal, Abbas Pirmoradi, Paul Thomas, Johanne R. Trippas, Ines Zelch, and Oleg Zendel. 2024. Report on the 8th Workshop on Search-Oriented Conversational Artificial Intelligence (SCAI 2024) at CHIIR 2024. SIGIR Forum 58, 1 (aug 2024), 1–12. https://doi.org/10.1145/3687273.3687282

Digital Library

[36]

Ruoyuan Gao and Chirag Shah. 2020. Toward creating a fairer ranking in search engine results. Information Processing & Management 57, 1 (2020), 102138. https://doi.org/10.1016/j.ipm.2019.102138

Digital Library

[37]

Marcel Gohsen, Johannes Kiesel, Mariam Korashi, Jan Ehlers, and Benno Stein. 2023. Guiding Oral Conversations: How to Nudge Users Towards Asking Questions?. In ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2023). ACM, New York, 34–42. https://doi.org/10.1145/3576840.3578291

Digital Library

[38]

Kylie L. Goodman and Christopher B. Mayhorn. 2023. It’s not what you say but how you say it: Examining the influence of perceived voice assistant gender and pitch on trust and reliance. Applied Ergonomics 106 (2023), 103864. https://doi.org/10.1016/j.apergo.2022.103864

[39]

Anthony G Greenwald, Debbie E McGhee, and Jordan LK Schwartz. 1998. Measuring Individual Differences in Implicit Cognition: The Implicit Association Test. Journal of personality and social psychology 74, 6 (1998), 1464.

[40]

Ido Guy. 2018. The Characteristics of Voice Search: Comparing Spoken with Typed-in Mobile Web Search Queries. ACM Trans. Inf. Syst. 36, 3, Article 30 (mar 2018), 28 pages. https://doi.org/10.1145/3182163

Digital Library

[41]

Jacek Gwizdka, Rahilsadat Hosseini, Michael Cole, and Shouyi Wang. 2017. Temporal Dynamics of Eye-Tracking and EEG During Reading and Relevance Decisions. Journal of the Association for Information Science and Technology 68, 10 (2017), 2299–2312. https://doi.org/10.1002/asi.23904

[42]

Christopher G. Harris. 2019. Detecting Cognitive Bias in a Relevance Assessment Task Using an Eye Tracker. In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications (Denver, Colorado) (ETRA ’19). Association for Computing Machinery, New York, NY, USA, Article 36, 5 pages. https://doi.org/10.1145/3314111.3319824

Digital Library

[43]

Danula Hettiachchi, Zhanna Sarsenbayeva, Fraser Allison, Niels van Berkel, Tilman Dingler, Gabriele Marini, Vassilis Kostakos, and Jorge Goncalves. 2020. "Hi! I am the Crowd Tasker" Crowdsourcing through Digital Voice Assistants. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376320

Digital Library

[44]

Kaixin Ji, Danula Hettiachchi, Flora D. Salim, Falk Scholer, and Damiano Spina. 2024. Characterizing Information Seeking Processes with Multiple Physiological Signals. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Washington, DC, USA) (SIGIR ’24). Association for Computing Machinery, New York, NY, USA, 12 pages. https://doi.org/10.1145/3626772.3657793

Digital Library

[45]

Kaixin Ji, Damiano Spina, Danula Hettiachchi, Flora Dylis Salim, and Falk Scholer. 2023. Towards Detecting Tonic Information Processing Activities with Physiological Data. In Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing (Cancun, Quintana Roo, Mexico) (UbiComp/ISWC ’23 Adjunct). Association for Computing Machinery, New York, NY, USA, 5 pages. https://doi.org/10.1145/3594739.3610679

Digital Library

[46]

Xiaoming Jiang, Kira Gossack-Keenan, and Marc D Pell. 2020. To believe or not to believe? How voice and accent information in speech alter listener impressions of trust. Quarterly Journal of Experimental Psychology 73, 1 (2020), 55–79.

[47]

Angel Jimenez-Molina, Cristian Retamal, and Hernan Lira. 2018. Using psychophysiological sensors to assess mental workload during web browsing. Sensors 18, 2 (2018), 458.

[48]

Daniel Kahneman. 2003. A Perspective on Judgment and Choice: Mapping Bounded Rationality.American Psychologist 58, 9 (2003), 697. https://doi.org/10.1037/0003-066X.58.9.697

[49]

Clare-Marie Karat, Christine Halverson, Daniel Horn, and John Karat. 1999. Patterns of entry and correction in large vocabulary continuous speech recognition systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, USA) (CHI ’99). Association for Computing Machinery, New York, NY, USA, 568–575. https://doi.org/10.1145/302979.303160

Digital Library

[50]

Anna Khofman. 2023. Exploring cognitive biases in voice-based virtual assistants.

[51]

Johannes Kiesel, Arefeh Bahrami, Benno Stein, Avishek Anand, and Matthias Hagen. 2018. Toward Voice Query Clarification. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (Ann Arbor, MI, USA) (SIGIR ’18). Association for Computing Machinery, New York, NY, USA, 1257–1260. https://doi.org/10.1145/3209978.3210160

Digital Library

[52]

Johannes Kiesel, Arefeh Bahrami, Benno Stein, Avishek Anand, and Matthias Hagen. 2019. Clarifying False Memories in Voice-Based Search. In Proceedings of the 2019 Conference on Human Information Interaction and Retrieval (Glasgow, Scotland UK) (CHIIR ’19). Association for Computing Machinery, New York, NY, USA, 331–335. https://doi.org/10.1145/3295750.3298961

Digital Library

[53]

Johannes Kiesel, Damiano Spina, Henning Wachsmuth, and Benno Stein. 2021. The Meant, the Said, and the Understood: Conversational Argument Search and Cognitive Biases. In Proceedings of the 3rd Conference on Conversational User Interfaces (Bilbao (online), Spain) (CUI ’21). Association for Computing Machinery, New York, NY, USA, Article 20, 5 pages. https://doi.org/10.1145/3469595.3469615

Digital Library

[54]

Sylvia D. Kreibig. 2010. Autonomic nervous system activity in emotion: A review. Biological Psychology 84, 3 (2010), 394–421. https://doi.org/10.1016/j.biopsycho.2010.03.010 The biopsychology of emotion: Current theoretical and empirical perspectives.

[55]

J. Satheesh Kumar and P. Bhuvaneswari. 2012. Analysis of Electroencephalography (EEG) Signals and Its Categorization–A Study. Procedia Engineering 38 (2012), 2525–2536. https://doi.org/10.1016/j.proeng.2012.06.298 International Conference on Modelling Optimization and Computing.

[56]

Pupil Labs. 2023. Neon: Eye tracking for research and beyond. https://pupil-labs.com/products/ Accessed: October 21, 2023.

[57]

Sunok Lee, Minji Cho, and Sangsu Lee. 2020. What If Conversational Agents Became Invisible? Comparing Users’ Mental Models According to Physical Entity of AI Speaker. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 3, Article 88 (sep 2020), 24 pages. https://doi.org/10.1145/3411840

Digital Library

[58]

Juan David Leongómez, Katarzyna Pisanski, David Reby, Disa Sauter, Nadine Lavan, Marcus Perlman, and Jaroslava Varella Valentova. 2021. Voice modulation: from origin and mechanism to social impact., 20200386 pages. https://doi.org/10.1098/rstb.2020.0386

[59]

Stephan Lewandowsky, KH Ecker Ullrich, and M Seifert Colleen. 2012. Misinformation and Its Correction: Continued Influence and Successful Debiasing. Psychological Science in the Public Interest 13, 3 (2012), 106–131. https://doi.org/10.1177/1529100612451018

[60]

Jason Edward Lewis, Angie Abdilla, Noelani Arista, Kaipulaumakaniolono Baker, Scott Benesiinaabandan, Michelle Brown, Melanie Cheung, Meredith Coleman, Ashley Cordes, Joel Davison, Kūpono Duncan, Sergio Garzon, D. Fox Harrell, Peter-Lucas Jones, Kekuhi Kealiikanakaoleohaililani, Megan Kelleher, Suzanne Kite, Olin Lagon, Jason Leigh, Maroussia Levesque, Keoni Mahelona, Caleb Moses, Isaac (’Ika’aka) Nahuewai, Kari Noe, Danielle Olson, ’Ōiwi Parker Jones, Caroline Running Wolf, Michael Running Wolf, Marlee Silva, Skawennati Fragnito, and Hēmi Whaanga. 2020. Indigenous Protocol and Artificial Intelligence Position Paper. (2020). https://doi.org/10.11573/SPECTRUM.LIBRARY.CONCORDIA.CA.00986506

[61]

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Proceedings of the 34th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS’20). Curran Associates Inc., Red Hook, NY, USA, Article 793, 16 pages.

Digital Library

[62]

Lizi Liao, Grace Hui Yang, and Chirag Shah. 2023. Proactive Conversational Agents in the Post-ChatGPT World. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (Taipei, Taiwan) (SIGIR ’23). Association for Computing Machinery, New York, NY, USA, 3452–3455. https://doi.org/10.1145/3539618.3594250

Digital Library

[63]

Mia Lindgren. 2023. Intimacy and emotions in podcast journalism: A study of award-winning Australian and British podcasts. Journalism Practice 17, 4 (2023), 704–719.

[64]

Steven J. Luck and Emily S. Kappenman. 2016. Electroencephalography and Event-Related Brain Potentials (4 ed.). Cambridge University Press, 74–100. https://doi.org/10.1017/9781107415782.005

[65]

Alexandre Marois, John E Marsh, and François Vachon. 2019. Is auditory distraction by changing-state and deviant sounds underpinned by the same mechanism? Evidence from pupillometry. Biological Psychology 141 (2019), 64–74.

[66]

Andrés Martínez-Maldonado, Rosa Jurado-Barba, Ana Sion, Isabel Domínguez-Centeno, Gabriela Castillo-Parra, Julio Prieto-Montalvo, and Gabriel Rubio. 2020. Brain functional connectivity after cognitive-bias modification and behavioral changes in abstinent alcohol-use disorder patients. International Journal of Psychophysiology 154 (2020), 46–58. https://doi.org/10.1016/j.ijpsycho.2019.10.004 Neurophysiologic impact of non-pharmacologic interventions for cognition.

[67]

David Maxwell and Leif Azzopardi. 2016. Agents, simulated users and humans: An analysis of performance and behaviour. In Proceedings of the 25th ACM international on conference on information and knowledge management. 731–740.

Digital Library

[68]

Shiri Melumad. 2023. Vocalizing search: How voice technologies alter consumer search processes and satisfaction. Journal of Consumer Research (2023), ucad009.

[69]

Miriam J Metzger and Andrew J Flanagin. 2013. Credibility and trust of information in online environments: The use of cognitive heuristics. Journal of pragmatics 59 (2013), 210–220.

[70]

Dominika Michalkova, Mario Parra-Rodriguez, and Yashar Moshfeghi. 2022. Information Need Awareness: An EEG Study. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (Madrid, Spain) (SIGIR ’22). Association for Computing Machinery, New York, NY, USA, 610–621. https://doi.org/10.1145/3477495.3531999

Digital Library

[71]

Randall K. Minas, Robert F. Potter, Alan R. Dennis, Valerie Bartelt, and Soyoung Bae. 2014. Putting on the Thinking Cap: Using NeuroIS to Understand Information Processing Biases in Virtual Teams. Journal of Management Information Systems 30, 4 (2014), 49–82. https://doi.org/10.2753/MIS0742-1222300403 arXiv:https://doi.org/10.2753/MIS0742-1222300403

[72]

Zainab Mohamed, Mohamed El Halaby, Tamer Said, Doaa Shawky, and Ashraf Badawi. 2018. Characterizing focused attention and working memory using EEG. Sensors 18, 11 (2018), 3743.

[73]

Santiago Morales and Maureen E. Bowers. 2022. Time-frequency analysis methods and their application in developmental EEG data. Developmental Cognitive Neuroscience 54 (2022), 101067. https://doi.org/10.1016/j.dcn.2022.101067

[74]

Patricia Moravec, Randall Minas, and Alan R Dennis. 2018. Fake news on social media: People believe what they want to believe when it makes no sense at all. Kelley School of Business research paper18-87 (2018).

[75]

Yashar Moshfeghi and Joemon M. Jose. 2013. An effective implicit relevance feedback technique using affective, physiological and behavioural features. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (Dublin, Ireland) (SIGIR ’13). Association for Computing Machinery, New York, NY, USA, 133–142. https://doi.org/10.1145/2484028.2484074

Digital Library

[76]

Yashar Moshfeghi, Peter Triantafillou, and Frank Pollick. 2019. Towards predicting a realisation of an information need based on brain signals. In The world wide web conference. 1300–1309.

[77]

Cosmin Munteanu and Gerald Penn. 2014. Speech-based interaction: myths, challenges, and opportunities. In Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices & Services (Toronto, ON, Canada) (MobileHCI ’14). Association for Computing Machinery, New York, NY, USA, 567–568. https://doi.org/10.1145/2628363.2645671

Digital Library

[78]

Muse. 2023. Muse EEG-Powered Meditation & Sleep Headband. https://choosemuse.com/ Accessed: October 21, 2023.

[79]

Chelsea Myers, Anushay Furqan, Jessica Nebolsky, Karina Caro, and Jichen Zhu. 2018. Patterns for How Users Overcome Obstacles in Voice User Interfaces. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3173574.3173580

Digital Library

[80]

Adit Nair, Roshan Mathew, Zihao Mei, Rezylle Milallos, Pratheep Kumar Chelladurai, and Tae Oh. 2023. Investigating the User Experience and Challenges of Food Delivery Applications for the Blind and Visually Impaired. In Proceedings of the 25th International Conference on Mobile Human-Computer Interaction (Athens, Greece) (MobileHCI ’23 Companion). Association for Computing Machinery, New York, NY, USA, Article 13, 6 pages. https://doi.org/10.1145/3565066.3608688

Digital Library

[81]

Yukiko I. Nakano and Ryo Ishii. 2010. Estimating User’s Engagement from Eye-Gaze Behaviors in Human-Agent Conversations. In Proceedings of the 15th International Conference on Intelligent User Interfaces (Hong Kong, China) (IUI ’10). Association for Computing Machinery, New York, NY, USA, 139–148. https://doi.org/10.1145/1719970.1719990

Digital Library

[82]

Alamir Novin and Eric Meyers. 2017. Making Sense of Conflicting Science Information: Exploring Bias in the Search Engine Result Page. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval (Oslo, Norway) (CHIIR ’17). Association for Computing Machinery, New York, NY, USA, 175–184. https://doi.org/10.1145/3020165.3020185

Digital Library

[83]

Julius Odede and Ingo Frommholz. 2024. JayBot – Aiding University Students and Admission with an LLM-based Chatbot. In Proceedings of the 2024 Conference on Human Information Interaction and Retrieval (Sheffield, United Kingdom) (CHIIR ’24). Association for Computing Machinery, New York, NY, USA, 391–395. https://doi.org/10.1145/3627508.3638293

Digital Library

[84]

Ryota Ooko, Ryo Ishii, and Yukiko I Nakano. 2011. Estimating a user’s conversational engagement based on head pose information. In Intelligent Virtual Agents: 10th International Conference, IVA 2011, Reykjavik, Iceland, September 15-17, 2011. Proceedings 11. Springer, 262–268. https://doi.org/10.1007/978-3-642-23974-8_29

[85]

Sachin Pathiyan Cherumanal. 2022. Fairness-Aware Question Answering for Intelligent Assistants(SIGIR ’22). Association for Computing Machinery, New York, NY, USA, 3492. https://doi.org/10.1145/3477495.3531682

Digital Library

[86]

Sachin Pathiyan Cherumanal, Damiano Spina, Falk Scholer, and W. Bruce Croft. 2021. Evaluating Fairness in Argument Retrieval. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (Virtual Event, Queensland, Australia) (CIKM ’21). Association for Computing Machinery, New York, NY, USA, 3363–3367. https://doi.org/10.1145/3459637.3482099

Digital Library

[87]

Sachin Pathiyan Cherumanal, Lin Tian, Futoon M. Abushaqra, Angel Felipe Magnossão de Paula, Kaixin Ji, Halil Ali, Danula Hettiachchi, Johanne R. Trippas, Falk Scholer, and Damiano Spina. 2024. Walert: Putting Conversational Information Seeking Knowledge into Action by Building and Evaluating a Large Language Model-Powered Chatbot. In Proceedings of the 2024 Conference on Human Information Interaction and Retrieval (Sheffield, United Kingdom) (CHIIR ’24). Association for Computing Machinery, New York, NY, USA, 401–405. https://doi.org/10.1145/3627508.3638309

Digital Library

[88]

Tam Pham, Zen Juen Lau, SH Annabel Chen, and Dominique Makowski. 2021. Heart rate variability in psychology: A review of HRV indices and an analysis tutorial. Sensors 21, 12 (2021), 3998. https://doi.org/10.3390/s21123998

[89]

Erin M. Picou, Todd A. Ricketts, and Benjamin WY. Hornsby. 2011. Visual cues and listening effort: Individual variability. Journal of Speech, Language, and Hearing Research 54, 5 (2011). https://doi.org/10.1044/1092-4388(2011/10-0154)

[90]

Gavin W Ploger, Johnanna Dunaway, Patrick Fournier, and Stuart Soroka. 2021. The psychophysiological correlates of cognitive dissonance. Politics and the Life Sciences 40, 2 (2021), 202–212. https://doi.org/10.1017/pls.2021.15

[91]

Filip Radlinski and Nick Craswell. 2017. A theoretical framework for conversational search. In Proceedings of the 2017 conference on conference human information interaction and retrieval. 117–126.

Digital Library

[92]

Stephen Rainey, Stéphanie Martin, Andy Christen, Pierre Mégevand, and Eric Fourneret. 2020. Brain recording, mind-reading, and neurotechnology: ethical issues from consumer devices to brain-based speech decoding. Science and engineering ethics 26 (2020), 2295–2311.

[93]

Mor Regev, Erez Simony, Katherine Lee, Kean Ming Tan, Janice Chen, and Uri Hasson. 2019. Propagation of information along the cortical hierarchy as a function of attention while reading and listening to stories. Cerebral Cortex 29, 10 (2019), 4017–4034.

[94]

Tobias Richter and Johanna Maier. 2017. Comprehension of multiple documents with conflicting information: A two-step model of validation. Educational Psychologist 52, 3 (2017), 148–166.

[95]

René Riedl, Fred D Davis, and Alan R Hevner. 2014. Towards a NeuroIS research methodology: intensifying the discussion on methods, tools, and measurement. Journal of the Association for Information Systems 15, 10 (2014), 4.

[96]

Ning Sa and Xiaojun Yuan. 2020. Examining users’ partial query modification patterns in voice search. Journal of the Association for Information Science and Technology 71, 3 (2020), 251–263.

Digital Library

[97]

Pascal Schneiders. 2020. What remains in mind? Effectiveness and efficiency of explainers at conveying information. Media and Communication 8, 1 (2020), 218–231.

[98]

Vidya Setlur and Melanie Tory. 2022. How do you Converse with an Analytical Chatbot? Revisiting Gricean Maxims for Designing Analytical Conversational Behavior. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 29, 17 pages. https://doi.org/10.1145/3491102.3501972

Digital Library

[99]

Chirag Shah and Emily M. Bender. 2022. Situating Search. In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval (Regensburg, Germany) (CHIIR ’22). Association for Computing Machinery, New York, NY, USA, 221–232. https://doi.org/10.1145/3498366.3505816

Digital Library

[100]

Nikhil Sharma, Q. Vera Liao, and Ziang Xiao. 2024. Generative Echo Chamber? Effects of LLM-Powered Search Systems on Diverse Information Seeking. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, 17 pages. https://doi.org/10.1145/3613904.3642459

Digital Library

[101]

Laurianne Sitbon, Gerd Berget, and Margot Brereton. 2023. Perspectives of Neurodiverse Participants in Interactive Information Retrieval. Found. Trends Inf. Retr. 17, 2 (jul 2023), 124–243. https://doi.org/10.1561/1500000086

Digital Library

[102]

Anastasia Smirnova. 2020. Word Order Communicates User Intent in Search Queries. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3375207

Digital Library

[103]

Evgeny N Sokolov, John A Spinks, Risto Näätänen, and Heikki Lyytinen. 2002. The orienting response in information processing.Lawrence Erlbaum Associates Publishers.

[104]

Michael Soprano, Kevin Roitero, David La Barbera, Davide Ceolin, Damiano Spina, Gianluca Demartini, and Stefano Mizzaro. 2024. Cognitive Biases in Fact-Checking and Their Countermeasures: A Review. Information Processing & Management 61, 3 (2024), 103672. https://doi.org/10.1016/j.ipm.2024.103672

Digital Library

[105]

Natarajan Sriram and Anthony G Greenwald. 2009. The Brief Implicit Association Test. Experimental psychology 56, 4 (2009), 283–294.

[106]

Masaki Suzuki and Yusuke Yamamoto. 2021. Characterizing the Influence of Confirmation Bias on Web Search Behavior. Frontiers in Psychology 12 (2021), 771948.

[107]

John Sweller. 2011. CHAPTER TWO - Cognitive Load Theory. Psychology of Learning and Motivation, Vol. 55. Academic Press, 37–76. https://doi.org/10.1016/B978-0-12-387691-1.00002-8

[108]

Madiha Tabassum, Tomasz Kosiński, Alisa Frik, Nathan Malkin, Primal Wijesekera, Serge Egelman, and Heather Richter Lipford. 2020. Investigating Users’ Preferences and Expectations for Always-Listening Voice Assistants. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 4, Article 153 (sep 2020), 23 pages. https://doi.org/10.1145/3369807

Digital Library

[109]

Johanne R. Trippas, Damiano Spina, Lawrence Cavedon, Hideo Joho, and Mark Sanderson. 2018. Informing the Design of Spoken Conversational Search: Perspective Paper. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (New Brunswick, NJ, USA) (CHIIR ’18). Association for Computing Machinery, New York, NY, USA, 32–41. https://doi.org/10.1145/3176349.3176387

Digital Library

[110]

Johanne R Trippas, Damiano Spina, Lawrence Cavedon, and Mark Sanderson. 2017. A conversational search transcription protocol and analysis. In Proc of sigir 1st international workshop on conversational approaches to information retrieval (cair’17), cair, Vol. 17.

[111]

Johanne R. Trippas, Damiano Spina, Lawrence Cavedon, and Mark Sanderson. 2017. How Do People Interact in Conversational Speech-Only Search Tasks: A Preliminary Analysis. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval (Oslo, Norway) (CHIIR ’17). Association for Computing Machinery, New York, NY, USA, 325–328. https://doi.org/10.1145/3020165.3022144

Digital Library

[112]

Johanne R. Trippas, Damiano Spina, Mark Sanderson, and Lawrence Cavedon. 2015. Results Presentation Methods for a Spoken Conversational Search System. In Proceedings of the First International Workshop on Novel Web Search Interfaces and Systems (Melbourne, Australia) (NWSearch ’15). Association for Computing Machinery, New York, NY, USA, 13–15. https://doi.org/10.1145/2810355.2810356

Digital Library

[113]

Johanne R. Trippas, Damiano Spina, Paul Thomas, Mark Sanderson, Hideo Joho, and Lawrence Cavedon. 2020. Towards a Model for Spoken Conversational Search. Information Processing & Management 57, 2 (2020), 102162. https://doi.org/10.1016/j.ipm.2019.102162

Digital Library

[114]

Kees Van den Bos. 2018. Why people radicalize: How unfairness judgments are used to fuel radical beliefs, extremist behaviors, and terrorism. Oxford University Press.

[115]

Yolanda Vazquez Alvarez and Stephen A. Brewster. 2010. Designing spatial audio interfaces to support multiple audio streams. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (Lisbon, Portugal) (MobileHCI ’10). Association for Computing Machinery, New York, NY, USA, 253–256. https://doi.org/10.1145/1851600.1851642

Digital Library

[116]

Alexandra Vtyurina, Denis Savenkov, Eugene Agichtein, and Charles LA Clarke. 2017. Exploring conversational search with humans, assistants, and wizards. In Proceedings of the 2017 chi conference extended abstracts on human factors in computing systems. 2187–2193.

Digital Library

[117]

Emily Wall, Leslie M Blaha, Lyndsey Franklin, and Alex Endert. 2017. Warning, bias may occur: A proposed approach to detecting cognitive bias in interactive visual analytics. In 2017 ieee conference on visual analytics science and technology (vast). IEEE, 104–115.

[118]

Jing Wei, Tilman Dingler, and Vassilis Kostakos. 2021. Developing the Proactive Speaker Prototype Based on Google Home. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, Article 292, 6 pages. https://doi.org/10.1145/3411763.3451642

Digital Library

[119]

Jing Wei, Tilman Dingler, and Vassilis Kostakos. 2022. Understanding User Perceptions of Proactive Smart Speakers. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 4, Article 185 (dec 2022), 28 pages. https://doi.org/10.1145/3494965

Digital Library

[120]

Jing Wei, Benjamin Tag, Johanne R Trippas, Tilman Dingler, and Vassilis Kostakos. 2022. What Could Possibly Go Wrong When Interacting with Proactive Smart Speakers? A Case Study Using an ESM Application. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 276, 15 pages. https://doi.org/10.1145/3491102.3517432

Digital Library

[121]

Andreas Wilke and Rui Mata. 2012. Cognitive bias. In Encyclopedia of human behavior. Academic Press, 531–535.

[122]

Yunhan Wu, Daniel Rough, Anna Bleakley, Justin Edwards, Orla Cooney, Philip R. Doyle, Leigh Clark, and Benjamin R. Cowan. 2020. See What I’m Saying? Comparing Intelligent Personal Assistant Use for Native and Non-Native Language Speakers. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (Oldenburg, Germany) (MobileHCI ’20). Association for Computing Machinery, New York, NY, USA, Article 34, 9 pages. https://doi.org/10.1145/3379503.3403563

Digital Library

[123]

Ziyi Ye, Xiaohui Xie, Yiqun Liu, Zhihong Wang, Xuesong Chen, Min Zhang, and Shaoping Ma. 2022. Towards a Better Understanding of Human Reading Comprehension with Brain Signals. In Proceedings of the ACM Web Conference 2022 (Virtual Event, Lyon, France) (WWW ’22). Association for Computing Machinery, New York, NY, USA, 380–391. https://doi.org/10.1145/3485447.3511966

Digital Library

[124]

Sofia Yfantidou, Pavlos Sermpezis, Athena Vakali, and Ricardo Baeza-Yates. 2023. Uncovering Bias in Personal Informatics. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 7, 3, Article 139 (sep 2023), 30 pages. https://doi.org/10.1145/3610914

Digital Library

[125]

Heeseung Yu and Eunkyoung Han. 2023. People see what they want to see: an EEG study. Cognitive Neurodynamics (2023), 1–15.

[126]

Tyson Yunkaporta. 2023. Right Story, Wrong Story: Adventures in Indigenous Thinking. Text Publishing Company.

[127]

Rafael Yuste. 2023. Advocating for neurodata privacy and neurotechnology regulation. Nature Protocols 18, 10 (2023), 2869–2875.

[128]

Hamed Zamani, Johanne R Trippas, Jeff Dalton, Filip Radlinski, 2023. Conversational information seeking. Foundations and Trends® in Information Retrieval 17, 3-4 (2023), 244–456.

[129]

Arne Freya Zillich and Lars Guenther. 2021. Selective Exposure to Information on the Internet: Measuring Cognitive Dissonance and Selective Exposure with Eye-Tracking. International Journal of Communication 15 (2021), 20.

Cited By

Pathiyan Cherumanal SScholer FTrippas JSpina D(2024)Towards Investigating Biases in Spoken Conversational SearchCompanion Proceedings of the 26th International Conference on Multimodal Interaction10.1145/3686215.3690156(61-66)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3686215.3690156

Index Terms

Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational Search
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Empirical studies in ubiquitous and mobile computing
2. Information systems
  1. Information retrieval
    1. Users and interactive retrieval

Recommendations

Towards a model for spoken conversational search
Highlights
- A first multi-level annotation schema for Spoken Conversational Search.
- ...
Abstract
Conversation is the natural mode for information exchange in daily life, a spoken conversational interaction for search input and output is a logical format for information seeking. However, the conceptualisation of user–system ...
Informing the Design of Spoken Conversational Search: Perspective Paper
CHIIR '18: Proceedings of the 2018 Conference on Human Information Interaction & Retrieval

We conducted a laboratory-based observational study where pairs of people performed search tasks communicating verbally. Examination of the discourse allowed commonly used interactions to be identified for Spoken Conversational Search (SCS). We compared ...
Results Presentation Methods for a Spoken Conversational Search System
NWSearch '15: Proceedings of the First International Workshop on Novel Web Search Interfaces and Systems

We propose research to investigate a new paradigm for Interactive Information Retrieval (IIR) where all input and output is mediated via speech. Our aim is to develop a new framework for effective and efficient IIR over a speech-only channel: a Spoken ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MobileHCI '24 Adjunct: Adjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction

September 2024

252 pages

ISBN:9798400705069

DOI:10.1145/3640471

Copyright © 2024 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 September 2024

Check for updates

Author Tags

Qualifiers

Extended-abstract
Research
Refereed limited

Funding Sources

Australian Research Council Centre of Excellence for Automated Decision-Making and Society
Australian Research Council

Conference

MobileHCI '24

Sponsor:

SIGCHI

MobileHCI '24: 26th International Conference on Mobile Human-Computer Interaction

September 30 - October 3, 2024

VIC, Melbourne, Australia

Acceptance Rates

Overall Acceptance Rate 202 of 906 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
103
Total Downloads

Downloads (Last 12 months)103
Downloads (Last 6 weeks)56

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pathiyan Cherumanal SScholer FTrippas JSpina D(2024)Towards Investigating Biases in Spoken Conversational SearchCompanion Proceedings of the 26th International Conference on Multimodal Interaction10.1145/3686215.3690156(61-66)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3686215.3690156

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents