Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3281505.3281514acmconferencesArticle/Chapter ViewAbstractPublication PagesvrstConference Proceedingsconference-collections
research-article

Sublime: a hands-free virtual reality menu navigation system using a high-frequency SSVEP-based brain-computer interface

Published: 28 November 2018 Publication History

Abstract

In this work we present Sublime, a new concept of Steady-State Visually Evoked Potential (SSVEP) based Brain-Computer Interface (BCI) where brain-computer communication occurs by capturing imperceptible visual stimuli integrated in the virtual scene and effortlessly conveying subliminal information to a computer. The technology was tested in a Virtual Reality (VR) environment, where the subject could navigate between the different menus by just gazing at them. The ratio between the stimuli frequencies and the refresh rate of the VR display creates an undesired perception of beats for which different solutions are proposed. To inform the user of target activation, real-time feedback in the form of loading bars is incorporated under each selectable object. We conducted experiments with several subjects and though the system is slower than a conventional joystick, users reported a satisfactory overall experience, in part due to the unexpected responsiveness of the system, as well as due to the fact that virtual objects flickered at a rate that did not cause annoyance. Since the imperceptible visual stimuli can be integrated unobtrusively to any element of the virtual world, we conclude that the potential applications of Sublime are extensive, especially in situations where knowing user's visual focus can be relevant.

References

[1]
Fabrizio Beverina, Giorgio Palmas, Stefano Silvoni, Francesco Piccione, Silvio Giove, et al. 2003. User adaptive BCIs: SSVEP and P300 based interfaces. Psych-Nology Journal 1, 4 (2003), 331--354.
[2]
Guangyu Bin, Xiaorong Gao, Zheng Yan, Bo Hong, and Shangkai Gao. 2009. An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. Journal of neural engineering 6, 4 (2009), 046002.
[3]
GS Brindley, JJ Du Croz, and WAH Rushton. 1966. The flicker fusion frequency of the blue-sensitive mechanism of colour vision. The Journal of physiology 183, 2 (1966), 497--500.
[4]
Josef Brozek and Ancel Keys. 1945. Changes in flicker-fusion frequency with age. Journal of Consulting Psychology 9, 2 (1945), 87.
[5]
Xiaogang Chen, Zhikai Chen, Shangkai Gao, and Xiaorong Gao. 2014. A high-itr ssvep-based bci speller. Brain-Computer Interfaces 1, 3--4 (2014), 181--191.
[6]
Xiaogang Chen, Yijun Wang, Masaki Nakanishi, Xiaorong Gao, Tzyy-Ping Jung, and Shangkai Gao. 2015. High-speed spelling with a noninvasive brain-computer interface. Proceedings of the national academy of sciences 112, 44 (2015), E6058--E6067.
[7]
Francis Crick and Christof Koch. 1995. Are we aware of neural activity in primary visual cortex? Nature 375, 6527 (1995), 121--123.
[8]
Stanley W Davis. 1955. Auditory and visual flicker-fusion as measures of fatigue. The American journal of psychology 68, 4 (1955), 654--657.
[9]
Pablo FDiez, Vicente A Mut, Enrique M Avila Perona, and Eric Laciar Leber. 2011. Asynchronous BCI control using high-frequency SSVEP. Journal of neuroengineering and rehabilitation 8, 1 (2011), 39.
[10]
Auria Eisen-Enosh, Nairouz Farah, Zvia Burgansky-Eliash, Uri Polat, and Yossi Mandel. 2017. Evaluation of Critical Flicker-Fusion Frequency Measurement Methods for the Investigation of Visual Temporal Resolution. Scientific reports 7, 1 (2017), 15621.
[11]
Josef Faller, Gernot Müller-Putz, Dieter Schmalstieg, and Gert Pfurtscheller. 2010. An application framework for controlling an avatar in a desktop-based virtual environment via a software SSVEP brain-computer interface. Presence: teleoperators and virtual environments 19, 1 (2010), 25--34.
[12]
Robert S Fisher, Graham Harding, Giuseppe Erba, Gregory L Barkley, and Arnold Wilkins. 2005. Photic-and pattern-induced seizures: a review for the Epilepsy Foundation of America Working Group. Epilepsia 46, 9 (2005), 1426--1441.
[13]
Xiaorong Gao, Dingfeng Xu, Ming Cheng, and Shangkai Gao. 2003. A BCI-based environmental controller for the motion-disabled. IEEE Transactions on neural systems and rehabilitation engineering 11, 2 (2003), 137--140.
[14]
Gary Garcia. 2008. High frequency SSVEPs for BCI applications. In Computer-Human Interaction. Citeseer.
[15]
Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139--183.
[16]
Christoph S Herrmann. 2001. Human EEG responses to 1--100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena. Experimental brain research 137, 3--4 (2001), 346--353.
[17]
Steven A Hillyard, Hermann Hinrichs, Claus Tempelmann, Stephen T Morgan, Jonathan C Hansen, Henning Scheich, and Hans-Jochen Heinze. 1997. Combining steady-state visual evoked potentials and f MRI to localize brain activity during selective attention. Human brain mapping 5, 4 (1997), 287--292.
[18]
Manling Huang, Pingdong Wu, Ying Liu, Luzheng Bi, and Hongwei Chen. 2008. Application and contrast in brain-computer interface Between hilbert-huang transform and wavelet transform. In Young Computer Scientists, 2008. ICYCS 2008. The 9th International Conference for. IEEE, 1706--1710.
[19]
Bonkon Koo, Hwan-Gon Lee, Yunjun Nam, and Seungjin Choi. 2015. Immersive BCI with SSVEP in VR head-mounted display. In Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE. IEEE, 1103--1106.
[20]
Carney Landis. 1954. Determinants of the critical flicker-fusion threshold. Physiological Reviews 34, 2 (1954), 259--286.
[21]
Anatole Lécuyer, Fabien Lotte, Richard B Reilly, Robert Leeb, Michitaka Hirose, and Mel Slater. 2008. Brain-computer interfaces, virtual reality, and videogames. Computer 41, 10 (2008).
[22]
Jozef Legény, Raquel Viciana Abad, and Anatole Lécuyer. 2011. Navigating in virtual worlds using a self-paced SSVEP-based brain-computer interface with integrated stimulation and real-time feedback. Presence: Teleoperators and Virtual Environments 20, 6 (2011), 529--544.
[23]
Zhonglin Lin, Changshui Zhang, Wei Wu, and Xiaorong Gao. 2007. Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. IEEE transactions on biomedical engineering 54, 6 (2007), 1172--1176.
[24]
Nikolay V Manyakov, Nikolay Chumerin, Arne Robben, Adrien Combaz, Marijn van Vliet, and Marc M Van Hulle. 2013. Sampled sinusoidal stimulation profile and multichannel fuzzy logic classification for monitor-based phase-coded SSVEP brain-computer interfacing. Journal of neural engineering 10, 3 (2013), 036011.
[25]
Ignas Martišius and Robertas Damaševičius. 2016. A prototype SSVEP based real time BCI gaming system. Computational intelligence and neuroscience 2016 (2016), 18.
[26]
Matlab. 2018. Retrieved March 2018 from https://www.mathworks.com/products/matlab.html
[27]
ST Morgan, JC Hansen, and SA Hillyard. 1996. Selective attention to stimulus location modulates the steady-state visual evoked potential. Proceedings of the National Academy of Sciences 93, 10 (1996), 4770--4774.
[28]
Matthias M Müller, P Malinowski, T Gruber, and SA Hillyard. 2003. Sustained division of the attentional spotlight. Nature 424, 6946 (2003), 309.
[29]
Matthias M Müller, Terence W Picton, Pedro Valdes-Sosa, Jorge Riera, Wolfgang A Teder-Sälejärvi, and Steven A Hillyard. 1998. Effects of spatial selective attention on the steady-state visual evoked potential in the 20--28 Hz range. Cognitive Brain Research 6, 4 (1998), 249--261.
[30]
OpenBCI. 2018. Retrieved March 2018 from http://openbci.com/
[31]
Maria APastor, Julio Artieda, Javier Arbizu, Miguel Valencia, and Jose C Masdeu. 2003. Human cerebral activation during steady-state visual-evoked responses. Journal of neuroscience 23, 37 (2003), 11621--11627.
[32]
Robert Prueckl and Christoph Guger. 2009. A brain-computer interface based on steady state visual evoked potentials for controlling a robot. In International Work-Conference on Artificial Neural Networks. Springer, 690--697.
[33]
David Regan. 1989. Human brain electrophysiology: evoked potentials and evoked magnetic fields in science and medicine. (1989).
[34]
Oculus Rift. 2018. Retrieved March 2018 from https://www.oculus.com/rift/
[35]
Frank Sharbrough. 1991. American Electroencephalographic Society guidelines for standard electrode position nomenclature. J clin Neurophysiol 8 (1991), 200--202.
[36]
Ernst Simonson and Josef Brozek. 1952. Flicker fusion frequency: background and applications. Physiological reviews 32, 3 (1952), 349--378.
[37]
Unity3D. 2018.
[38]
Yijun Wang, Ruiping Wang, Xiaorong Gao, Bo Hong, and Shangkai Gao. 2006. A practical VEP-based brain-computer interface. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14, 2 (2006), 234--240.
[39]
Jonathan R Wolpaw, Niels Birbaumer, Dennis J McFarland, Gert Pfurtscheller, and Theresa M Vaughan. 2002. Brain-computer interfaces for communication and control. Clinical neurophysiology 113, 6 (2002), 767--791.
[40]
Wang Yijun, Wang Ruiping, Gao Xiaorong, and Gao Shangkai. 2005. Brain-computer interface based on the high-frequency steady-state visual evoked potential. In Neural Interface and Control, 2005. Proceedings. 2005 First International Conference on. IEEE, 37--39.
[41]
Danhua Zhu, Jordi Bieger, Gary Garcia Molina, and Ronald M Aarts. 2010. A survey of stimulation methods used in SSVEP-based BCIs. Computational intelligence and neuroscience 2010 (2010), 1.

Cited By

View all
  • (2024)Advancements in brain-computer interfaces for the rehabilitation of unilateral spatial neglect: a concise reviewFrontiers in Neuroscience10.3389/fnins.2024.137337718Online publication date: 9-May-2024
  • (2024)VR-SSVEPeripheral: Designing Virtual Reality Friendly SSVEP Stimuli using Peripheral Vision Area for Immersive and Comfortable ExperienceExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651084(1-7)Online publication date: 11-May-2024
  • (2023)Brain-Hack: Remotely Injecting False Brain-Waves with RF to Take Control of a Brain-Computer InterfaceProceedings of the 5th Workshop on CPS&IoT Security and Privacy10.1145/3605758.3623497(53-66)Online publication date: 26-Nov-2023
  • Show More Cited By

Index Terms

  1. Sublime: a hands-free virtual reality menu navigation system using a high-frequency SSVEP-based brain-computer interface

        Recommendations

        Comments

        Please enable JavaScript to view thecomments powered by Disqus.

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology
        November 2018
        570 pages
        ISBN:9781450360869
        DOI:10.1145/3281505
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 28 November 2018

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. brain-computer interface
        2. electroencephalography
        3. steady-state visually evoked potentials
        4. virtual reality

        Qualifiers

        • Research-article

        Conference

        VRST '18

        Acceptance Rates

        Overall Acceptance Rate 66 of 254 submissions, 26%

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)55
        • Downloads (Last 6 weeks)8
        Reflects downloads up to 04 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Advancements in brain-computer interfaces for the rehabilitation of unilateral spatial neglect: a concise reviewFrontiers in Neuroscience10.3389/fnins.2024.137337718Online publication date: 9-May-2024
        • (2024)VR-SSVEPeripheral: Designing Virtual Reality Friendly SSVEP Stimuli using Peripheral Vision Area for Immersive and Comfortable ExperienceExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651084(1-7)Online publication date: 11-May-2024
        • (2023)Brain-Hack: Remotely Injecting False Brain-Waves with RF to Take Control of a Brain-Computer InterfaceProceedings of the 5th Workshop on CPS&IoT Security and Privacy10.1145/3605758.3623497(53-66)Online publication date: 26-Nov-2023
        • (2023)A Systematic Review of Interaction Approaches based on Visually Evoked PotentialsProceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3594806.3594862(396-401)Online publication date: 5-Jul-2023
        • (2023)EEG-Based Brain-Computer Interactions in Immersive Virtual and Augmented Reality: A Systematic ReviewProceedings of the ACM on Human-Computer Interaction10.1145/35932267:EICS(1-33)Online publication date: 19-Jun-2023
        • (2023)Neural Applications Using Immersive Virtual Reality: A Review on EEG StudiesIEEE Transactions on Neural Systems and Rehabilitation Engineering10.1109/TNSRE.2023.325455131(1645-1658)Online publication date: 2023
        • (2023)TRCA-Net: using TRCA filters to boost the SSVEP classification with convolutional neural networkJournal of Neural Engineering10.1088/1741-2552/ace38020:4(046005)Online publication date: 12-Jul-2023
        • (2022)Novel hybrid visual stimuli incorporating periodic motions into conventional flickering or pattern-reversal visual stimuli for steady-state visual evoked potential-based brain-computer interfacesFrontiers in Neuroinformatics10.3389/fninf.2022.99706816Online publication date: 21-Sep-2022
        • (2022)Exploring the effects of head movements and accompanying gaze fixation switch on steady-state visual evoked potentialFrontiers in Human Neuroscience10.3389/fnhum.2022.94307016Online publication date: 12-Sep-2022
        • (2022)The Butterfly Effect: Novel Opportunities for Steady-State Visually-Evoked Potential Stimuli in Virtual RealityProceedings of the Augmented Humans International Conference 202210.1145/3519391.3519397(254-266)Online publication date: 13-Mar-2022
        • Show More Cited By

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media