panel

NeuroCHI: Are We Prepared for the Integration of the Brain with Computing?

Authors:

Robert J.K Jacob,

Pedro LopesAuthors Info & Claims

CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

Article No.: 588, Pages 1 - 5

https://doi.org/10.1145/3613905.3643973

Published: 11 May 2024 Publication History

Abstract

Recent advancements in neuroscience, wearable technology, and artificial intelligence are paving the way for computing systems that are integrated with the brain and nervous system. Over the past years, we have witnessed the simultaneous progression of wearable neurotechnology and AI-based modeling and analysis of brain data. Coincidentally, this period has also seen HCI researchers showcase their translational work, incorporating neuroscience insights into innovative interactive systems, including brain-computer interfaces (BCIs) and non-invasive brain stimulation. These efforts may transform our brain and nervous system activity into direct interfaces for interacting with computing systems. Our panel poses the question: "Is the HCI community ready for the integration of the brain with computing?" Together with a panel of experts, we will review the current state of the intersection between HCI and neurotechnology, discuss the research questions and novel applications that emerge from merging these two fields, and debate ethical implications.

References

[1]

[1] 2024. https://blueberryx.com/pages/team

[2]

Judith Amores, Anna Fuste, and Robert Richer. 2019. Deep reality: Towards increasing relaxation in VR by Subtly changing light, sound and movement based on HR, EDA, and EEG. In Extended abstracts of the 2019 CHI conference on human factors in computing systems. 1–2.

[3]

Daniel Anaya, Gautam Batra, Peter Bracewell, Ryan Catoen, Dev Chakraborty, Mark Chevillet, Pradeep Damodara, Alvin Dominguez, Laurence Emms, Zifan Jiang, 2023. Scalable, modular continuous wave functional near-infrared spectroscopy system (Spotlight). Journal of Biomedical Optics 28, 6 (2023), 065003–065003.

[4]

[4] Inside Facebook Reality Labs: Wrist based interaction for the next computing platform. 2021. https://tech.facebook.com/reality-labs/2021/3/inside-facebook-reality-labs-wrist-based-interaction-for-the-next-computing-platform/

[5]

Guillermo Bernal, Nelson Hidalgo, Conor Russomanno, and Pattie Maes. 2022. Galea: A physiological sensing system for behavioral research in Virtual Environments. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 66–76.

[6]

Alexander J Casson. 2019. Wearable EEG and beyond. Biomedical engineering letters 9, 1 (2019), 53–71.

[7]

Valdemar Danry, Laura Chicos, Matheus Fonseca, Ishraki Kazi, and Pattie Maes. 2024. Synthetic Visual Sensations: Augmenting Human Spatial Awareness with a Wearable Retinal Electric Stimulation Device. In Proceedings of the Augmented Humans International Conference 2024.

[8]

Eamon Doherty, Gilbert Cockton, Chris Bloor, and Dennis Benigno. 2001. Improving the performance of the cyberlink mental interface with “yes/no program”. In Proceedings of the SIGCHI conference on Human factors in computing systems. 69–76.

Digital Library

[9]

Xiao Fang, Nathan Semertzidis, Michaela Scary, Xinyi Wang, Josh Andres, Fabio Zambetta, and Florian’Floyd’ Mueller. 2021. Telepathic Play: Towards Playful Experiences Based on Brain-to-brain Interfacing. In Extended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play. 268–273.

[10]

Sharlene N Flesher, Jennifer L Collinger, Stephen T Foldes, Jeffrey M Weiss, John E Downey, Elizabeth C Tyler-Kabara, Sliman J Bensmaia, Andrew B Schwartz, Michael L Boninger, and Robert A Gaunt. 2016. Intracortical microstimulation of human somatosensory cortex. Science translational medicine 8, 361 (2016), 361ra141–361ra141.

[11]

Matteo Franza, Giuliana Sorrentino, Matteo Vissani, Andrea Serino, Olaf Blanke, and Michela Bassolino. 2019. Hand perceptions induced by single pulse transcranial magnetic stimulation over the primary motor cortex. Brain stimulation 12, 3 (2019), 693–701.

[12]

Mariam Hassib, Stefan Schneegass, Philipp Eiglsperger, Niels Henze, Albrecht Schmidt, and Florian Alt. 2017. EngageMeter: A System for Implicit Audience Engagement Sensing Using Electroencephalography. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). Association for Computing Machinery, New York, NY, USA, 5114–5119. https://doi.org/10.1145/3025453.3025669

Digital Library

[13]

Adam Haar Horowitz, Kathleen Esfahany, Tomás Vega Gálvez, Pattie Maes, and Robert Stickgold. 2023. Targeted dream incubation at sleep onset increases post-sleep creative performance. Scientific reports 13, 1 (2023), 7319.

[14]

Fatma Inanici, Lorie N Brighton, Soshi Samejima, Christoph P Hofstetter, and Chet T Moritz. 2021. Transcutaneous spinal cord stimulation restores hand and arm function after spinal cord injury. IEEE Transactions on Neural Systems and Rehabilitation Engineering 29 (2021), 310–319.

[15]

Abhinandan Jain, Felix Schoeller, Emilie Zhang, and Pattie Maes. 2022. Frisson: Leveraging metasomatic interactions for generating aesthetic chills. In Proceedings of the 2022 International Conference on Multimodal Interaction. 148–158.

Digital Library

[16]

Nataliya Kosmyna and Pattie Maes. 2019. AttentivU: an EEG-based closed-loop biofeedback system for real-time monitoring and improvement of engagement for personalized learning. Sensors 19, 23 (2019), 5200.

[17]

Eike Langbehn, Frank Steinicke, Ping Koo-Poeggel, Lisa Marshall, and Gerd Bruder. 2019. Stimulating the brain in VR: Effects of transcranial direct-current stimulation on redirected walking. In ACM Symposium on Applied Perception 2019. 1–9.

Digital Library

[18]

Johnny Chung Lee and Desney S. Tan. 2006. Using a low-cost electroencephalograph for task classification in HCI research. In Proceedings of the 19th annual ACM symposium on User interface software and technology - UIST ’06. ACM Press, Montreux, Switzerland, 81. https://doi.org/10.1145/1166253.1166268

Digital Library

[19]

Bigna Lenggenhager, Tej Tadi, Thomas Metzinger, and Olaf Blanke. 2007. Video ergo sum: manipulating bodily self-consciousness. Science 317, 5841 (2007), 1096–1099.

[20]

Pedro Lopes, Lewis L Chuang, and Pattie Maes. 2021. Physiological I/O. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1–4.

Digital Library

[21]

Pedro Lopes, Sijing You, Lung-Pan Cheng, Sebastian Marwecki, and Patrick Baudisch. 2017. Providing haptics to walls & heavy objects in virtual reality by means of electrical muscle stimulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 1471–1482.

Digital Library

[22]

Christophe Lopez, Olaf Blanke, and FW Mast. 2012. The human vestibular cortex revealed by coordinate-based activation likelihood estimation meta-analysis. Neuroscience 212 (2012), 159–179.

[23]

Anton Nijholt, Robert JK Jacob, Marvin Andujar, Beste F Yuksel, and Grace Leslie. 2018. Brain-computer interfaces for artistic expression. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. 1–7.

Digital Library

[24]

Shuto Ogihara, Tomohiro Amemiya, Hideaki Kuzuoka, Takuji Narumi, and Kazuma Aoyama. 2023. Multi Surface Electrodes Nerve Bundles Stimulation on the Wrist: Modified Location of Tactile Sensation on the Palm. IEEE Access 11 (2023), 13794–13809.

[25]

[25] Muse Home Page. 2023. https://choosemuse.com/

[26]

[26] Neurable Home Page. 2023. https://neurable.com/

[27]

Paola Pinti, Clarisse Aichelburg, Sam Gilbert, Antonia Hamilton, Joy Hirsch, Paul Burgess, and Ilias Tachtsidis. 2018. A review on the use of wearable functional near-infrared spectroscopy in naturalistic environments. Japanese Psychological Research 60, 4 (2018), 347–373.

[28]

Janine Reis, Orlando B Swayne, Yves Vandermeeren, Mickael Camus, Michael A Dimyan, Michelle Harris-Love, Monica A Perez, Patrick Ragert, John C Rothwell, and Leonardo G Cohen. 2008. Contribution of transcranial magnetic stimulation to the understanding of cortical mechanisms involved in motor control. The Journal of physiology 586, 2 (2008), 325–351.

[29]

Tobias Röddiger, Christopher Clarke, Paula Breitling, Tim Schneegans, Haibin Zhao, Hans Gellersen, and Michael Beigl. 2022. Sensing with earables: A systematic literature review and taxonomy of phenomena. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1–57.

Digital Library

[30]

Simone Rossi, Mark Hallett, Paolo M Rossini, Alvaro Pascual-Leone, Safety of TMS Consensus Group, 2009. Safety, ethical considerations, and application guidelines for the use of transcranial magnetic stimulation in clinical practice and research. Clinical neurophysiology 120, 12 (2009), 2008–2039.

[31]

Filip Škola and Fotis Liarokapis. 2019. Examining and enhancing the illusory touch perception in virtual reality using non-invasive brain stimulation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[32]

Erin Solovey, Paul Schermerhorn, Matthias Scheutz, Angelo Sassaroli, Sergio Fantini, and Robert Jacob. 2012. Brainput: enhancing interactive systems with streaming fnirs brain input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’12). Association for Computing Machinery, New York, NY, USA, 2193–2202. https://doi.org/10.1145/2207676.2208372

Digital Library

[33]

Misha Sra, Abhinandan Jain, and Pattie Maes. 2019. Adding proprioceptive feedback to virtual reality experiences using galvanic vestibular stimulation. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–14.

Digital Library

[34]

Pamela Svensson, Ulrika Wijk, Anders Björkman, and Christian Antfolk. 2017. A review of invasive and non-invasive sensory feedback in upper limb prostheses. Expert review of medical devices 14, 6 (2017), 439–447.

[35]

Yudai Tanaka, Jacob Serfaty, and Pedro Lopes. 2024. Haptic Source-Effector: Full-Body Haptics via Non-Invasive Brain Stimulation. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642483

Digital Library

[36]

Yudai Tanaka, Alan Shen, Andy Kong, and Pedro Lopes. 2023. Full-hand Electro-Tactile Feedback without Obstructing Palmar Side of Hand. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15. https://doi.org/10.1145/3544548.3581382

Digital Library

[37]

Yudai Tanaka, Akifumi Takahashi, and Pedro Lopes. 2023. Interactive Benefits from Switching Electrical to Magnetic Muscle Stimulation. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–12.

Digital Library

[38]

Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander G Huth. 2023. Semantic reconstruction of continuous language from non-invasive brain recordings. Nature Neuroscience (2023), 1–9.

[39]

Erin Treacy Solovey, Daniel Afergan, Evan M. Peck, Samuel W. Hincks, and Robert J. K. Jacob. 2015. Designing Implicit Interfaces for Physiological Computing: Guidelines and Lessons Learned Using fNIRS. ACM Transactions on Computer-Human Interaction 21, 6 (Jan. 2015), 35:1–35:27. https://doi.org/10.1145/2687926

Digital Library

[40]

Boris M Velichkovsky and John Paulin Hansen. 1996. New technological windows into mind: There is more in eyes and brains for human-computer interaction. In Proceedings of the SIGCHI conference on Human factors in computing systems. 496–503.

Digital Library

[41]

Chi Vi and Sriram Subramanian. 2012. Detecting error-related negativity for interaction design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’12). Association for Computing Machinery, New York, NY, USA, 493–502. https://doi.org/10.1145/2207676.2207744

Digital Library

[42]

Athanasios Vourvopoulos, Evangelos Niforatos, and Michail Giannakos. 2019. EEGlass: An EEG-eyeware prototype for ubiquitous brain-computer interaction. In Adjunct proceedings of the 2019 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2019 ACM international symposium on wearable computers. 647–652.

Digital Library

[43]

Angela Vujic, Shreyas Nisal, and Pattie Maes. 2023. Joie: a Joy-based Brain-Computer Interface (BCI). In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–14.

Digital Library

[44]

Anna Wexler and Peter B Reiner. 2019. Oversight of direct-to-consumer neurotechnologies. Science 363, 6424 (2019), 234–235.

[45]

Nathan W Whitmore and Ken A Paller. 2023. Sleep disruption by memory cues selectively weakens reactivated memories. Learning & Memory 30, 3 (2023), 63–69.

[46]

Max L Wilson, Serena Midha, Horia A Maior, Anna L Cox, Lewis L Chuang, and Lachlan D Urquhart. 2022. SIG: Moving from Brain-Computer Interfaces to Personal Cognitive Informatics. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–4.

Digital Library

[47]

Tongda Xu, Dinglu Wang, and Xiaohui You. 2018. MindGame: Mediating people’s EEG alpha band power through reinforcement learning. In Adjunct Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 5–6.

Digital Library

[48]

Beste F. Yuksel, Kurt B. Oleson, Lane Harrison, Evan M. Peck, Daniel Afergan, Remco Chang, and Robert JK Jacob. 2016. Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty Based on Brain State. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI ’16). Association for Computing Machinery, New York, NY, USA, 5372–5384. https://doi.org/10.1145/2858036.2858388

Digital Library

[49]

Rafael Yuste, Sara Goering, Blaise Agüeray Arcas, Guoqiang Bi, Jose M. Carmena, Adrian Carter, Joseph J. Fins, Phoebe Friesen, Jack Gallant, Jane E. Huggins, Judy Illes, Philipp Kellmeyer, Eran Klein, Adam Marblestone, Christine Mitchell, Erik Parens, Michelle Pham, Alan Rubel, Norihiro Sadato, Laura Specker Sullivan, Mina Teicher, David Wasserman, Anna Wexler, Meredith Whittaker, and Jonathan Wolpaw. 2017. Four ethical priorities for neurotechnologies and AI. Nature 551, 7679 (2017), 159–163. https://doi.org/10.1038/551159a

Cited By

Tanaka Y(2024)Nervous System Interception: A New Paradigm for HapticsAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686715(1-5)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686715
Nguyen MYeo KSamaradivakara YNanayakkara S(2024)Catch that butterfly: A Multimodal Approach for Detecting and Simulating Gut FeelingsAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686340(1-3)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686340

Index Terms

NeuroCHI: Are We Prepared for the Integration of the Brain with Computing?
1. Hardware
  1. Emerging technologies
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Towards ambulatory brain-computer interfaces: a pilot study with P300 signals
ACE '09: Proceedings of the International Conference on Advances in Computer Entertainment Technology

Brain-Computer Interfaces (BCI) are communication systems that enable users to interact with computers using only brain activity. This activity is generally measured by ElectroEncephaloGraphy (EEG). A major limitation of BCI is the electrical ...
An auditory oddball based brain-computer interface system using multivariate EMD
ICIC'10: Proceedings of the Advanced intelligent computing theories and applications, and 6th international conference on Intelligent computing

A brain-computer interface (BCI) is a communication system that allows users to act on their environment by using only brain-activity. This paper presents a novel design of the auditory oddball task based brain-computer interface (BCI) system. The ...
Feature Analysis of EEG Based Brain-Computer Interfaces to Detect Motor Imagery
Brain Informatics
Abstract
Brain-Computer Interfaces (BCI) is one of the alluring breakthroughs for mankind as it provides a new way of communication for the patients of neuro-muscular disorders. Electroencephalography (EEG) signals are the most studied type of signals to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

May 2024

4761 pages

ISBN:9798400703317

DOI:10.1145/3613905

Editors:
Florian Floyd Mueller
Monash University
,
Penny Kyburz
The Australian National University
,
Julie R. Williamson
University of Glasgow
,
Corina Sas
Lancaster University

Copyright © 2024 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 May 2024

Check for updates

Author Tags

Qualifiers

Panel
Research
Refereed limited

Funding Sources

NSF (National Science Foundation)

Conference

CHI '24

Sponsor:

CHI '24: CHI Conference on Human Factors in Computing Systems

May 11 - 16, 2024

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
596
Total Downloads

Downloads (Last 12 months)596
Downloads (Last 6 weeks)87

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tanaka Y(2024)Nervous System Interception: A New Paradigm for HapticsAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686715(1-5)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686715
Nguyen MYeo KSamaradivakara YNanayakkara S(2024)Catch that butterfly: A Multimodal Approach for Detecting and Simulating Gut FeelingsAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686340(1-3)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686340

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Table of Contents