research-article

Open access

A brain-computer interface for high-level remote control of an autonomous, reinforcement-learning-based robotic system for reaching and grasping

Authors:

Lukas D.J. Fiederer,

Martin Voelker,

Alexander Knorr,

Martin Riedmiller,

Tonio BallAuthors Info & Claims

IUI '14: Proceedings of the 19th international conference on Intelligent User Interfaces

Pages 83 - 88

https://doi.org/10.1145/2557500.2557533

Published: 24 February 2014 Publication History

Abstract

We present an Internet-based brain-computer interface (BCI) for controlling an intelligent robotic device with autonomous reinforcement-learning. BCI control was achieved through dry-electrode electroencephalography (EEG) obtained during imaginary movements. Rather than using low-level direct motor control, we employed a high-level control scheme of the robot, acquired via reinforcement learning, to keep the users cognitive load low while allowing control a reaching-grasping task with multiple degrees of freedom. High-level commands were obtained by classification of EEG responses using an artificial neural network approach utilizing time-frequency features and conveyed through an intuitive user interface. The novel ombination of a rapidly operational dry electrode setup, autonomous control and Internet connectivity made it possible to conveniently interface subjects in an EEG laboratory with remote robotic devices in a closed-loop setup with online visual feedback of the robots actions to the subject. The same approach is also suitable to provide home-bound patients with the possibility to control state-of-the-art robotic devices currently confined to a research environment. Thereby, our BCI approach could help severely paralyzed patients by facilitating patient-centered research of new means of communication, mobility and independence.

References

[1]

Allison, B. Z., Leeb, R., Brunner, C., Müller-Putz, G. R., Bauernfeind, G., Kelly, J. W., and Neuper, C. Toward smarter BCIs: extending BCIs through hybridization and intelligent control. Journal of Neural Engineering 9 (2012), 1.

[2]

Brunner, C., Naeem, M., Leeb, R., Graimann, B., and Pfurtscheller, G. Spatial filtering and selection of optimized components in four class motor imagery EEG data using independent components analysis. Pattern Recognition Letters 28, 8 (2007), 957--964.

Digital Library

[3]

Bryan, M., Green, J., Chung, M., Chang, L. Y., Scherer, R., Smith, J. R., and Rao, R. P. N. An adaptive brain-computer interface for humanoid robot control. In Humanoids (2011), 199--204.

[4]

Collinger, J. L., Wodlinger, B., Downey, J. E., Wang, W., Tyler-Kabara, E. C., Weber, D. J., McMorland, A. J. C., Velliste, M., Boninger, M. L., and Schwartz, A. B. High-performance neuroprosthetic control by an individual with tetraplegia. The Lancet 381, 9866 (2013), 557--564.

[5]

Escolano, C., Antelis, J., and Minguez, J. Human brain-teleoperated robot between remote places. In Robotics and Automation, 2009. ICRA '09. IEEE International Conference on (2009), 4430--4437.

Digital Library

[6]

Ferreira, A., Bastos-Filho, T., Sarcinelli-Filho, M., Cheein, F., Postigo, J., and Carelli, R. Teleoperation of an industrial manipulator through a tcp/ip channel using eeg signals. In Industrial Electronics, 2006 IEEE International Symposium on, vol. 4 (2006), 3066--3071.

[7]

Hochberg, L. R., Bacher, D., Jarosiewicz, B., Masse, N. Y., Simeral, J. D., Vogel, J., Haddadin, S., Cash, J. L. S. S., van der Smagt, P., and Donoghue, J. P. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 7398 (2012), 372--375.

[8]

Lampe, T., and Riedmiller, M. Acquiring visual servoing reaching and grasping skills using neural reinforcement learning. IEEE International Joint Conference on Neural Networks (IJCNN) (2013).

[9]

Lotte, F., Congedo, M., Lcuyer, A., Lamarche, F., and Arnaldi, B. A review of classification algorithms for EEG-based brain-computer interfaces. Journal of Neural Engineering 4, 2 (2007), R1--R13.

[10]

MacKay, D. J. C. A practical bayesian framework for backpropagation networks. Neural Computation 4, 3 (1992), 448--472.

Digital Library

[11]

Moran, D. Evolution of brain-computer interface: action potentials, local field potentials and electrocorticograms. Current Opinions in Neurobiology 20, 6 (2010), 741--745.

[12]

Musallam, S., Corneil, B. D., Greger, B., Scherberger, H., and Andersen, R. A. Cognitive control signals for neural prosthetics. Science 305 (2004), 258--262.

[13]

Neuper, C., Müller-Putz, G. R., Scherer, R., and Pfurtscheller, G. Motor imagery and EEG-based control of spelling devices and neuroprostheses. Progress in Brain Research 159 (2006), 393--409.

[14]

Oostenveld, R., and Praamstra, P. The five percent electrode system for high-resolution EEG and ERP measurements. Clinical Neurophysiology 112, 4 (2001), 713--719.

[15]

Pfurtscheller, G., and F.H. Lopes da Silva. Event-related EEG/MEG synchronization and desynchronization: basic principles. Clinical Neurophysiology 110, 11 (1999), 1842--1857.

[16]

Riedmiller, M. Neural fitted Q iteration -- first experiences with a data efficient neural reinforcement learning method. In European Conference on Machine Learning, Springer (2005), 317--328.

Digital Library

[17]

Schalk, G., Mcfarl, D. J., Hinterberger, T., Birbaumer, N., and Wolpaw, J. R. BCI2000: A general-purpose brain-computer interface (BCI) system. IEEE Transactions on Biomedical Engineering 51 (2004), 2004.

[18]

Sutton, R. S., and Barto, A. G. Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). A Bradford Book, 1998.

Digital Library

[19]

Thomson, D. Spectrum estimation and harmonic analysis. Proceedings of the IEEE 70, 9 (1982), 1055--1096.

[20]

Vidaurre, C., Kawanabe, M., von Bünau, P., Blankertz, B., and Müller, K. R. Toward unsupervised adaptation of LDA for brain-computer interfaces. IEEE Transactions on Biomedical Engineering 58, 3 (2011), 587--597.

[21]

Vidaurre, C., Sannelli, C., Müller, K. R., and Blankertz, B. Machine-learning-based coadaptive calibration for brain-computer interfaces. Neural Computation 16 (2010).

Digital Library

[22]

Waldert, S., Pistohl, T., Braun, C., Ball, T., Aertsen, A., and Mehring, C. A review on directional information in neural signals for brain-machine interfaces. Journal of Physiology -- Paris 103, 3--5 (2009), 244--254.

[23]

Wang, D., Miao, D., and Blohm, G. Multi-class motor imagery EEG decoding for brain-computer interfaces. Frontiers in Neuroprosthetics 6 (2012), 151.

Cited By

Yu Yxia BXie MLi ZWang X(2024)Dynamic Modeling for Reinforcement Learning with Random DelayArtificial Neural Networks and Machine Learning – ICANN 202410.1007/978-3-031-72341-4_26(381-396)Online publication date: 17-Sep-2024
https://doi.org/10.1007/978-3-031-72341-4_26
Agrawal SMitra P(2024)Deep Reinforcement Learning in Healthcare and Biomedical ResearchDeep Reinforcement Learning and Its Industrial Use Cases10.1002/9781394272587.ch9(179-205)Online publication date: 4-Oct-2024
https://doi.org/10.1002/9781394272587.ch9
Che NZhang TLi YYu FWang H(2023)RLSF: Multimodal Sleep Improvement Based Reinforcement LearningIEEE Access10.1109/ACCESS.2023.326609411(47712-47724)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3266094
Show More Cited By

Index Terms

A brain-computer interface for high-level remote control of an autonomous, reinforcement-learning-based robotic system for reaching and grasping
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Towards Hybrid Multimodal Brain Computer Interface for Robotic Arm Command
Augmented Cognition
Abstract
A hybrid brain-computer interface (BCI) is a system that combines multiple biopotentials or different types of devices with a typical BCI system to enhance the interaction paradigms and various functional parameters. In this paper we present the ...
Shared control of a robotic arm using non-invasive brain–computer interface and computer vision guidance
Abstract
Control of a robotic arm using a brain–computer interface (BCI) for reach and grasp activities is one of the most fascinating applications for some severely disabled people, which is especially challenging for the non-invasive BCIs ...
Discriminating the brain activities for brain---computer interface applications through the optimal allocation-based approach

The translation of brain activities into signals in brain---computer interface (BCI) systems requires a robust and accurate classification to develop a communication system for motor disabled people. In BCIs, motor imagery (MI) tasks generate brain ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '14: Proceedings of the 19th international conference on Intelligent User Interfaces

February 2014

386 pages

ISBN:9781450321846

DOI:10.1145/2557500

General Chairs:
Tsvi Kuflik
The University of Haifa, Israel
,
Oliviero Stock
FBK-IRST, Italy
,
Program Chairs:
Joyce Chai
Michigan State University
,
Antonio Krüger
Saarland University and DFKI GmbH, Germany

Copyright © 2014 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 February 2014

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IUI'14

Sponsor:

IUI'14: IUI'14 19th International Conference on Intelligent User Interfaces

February 24 - 27, 2014

Haifa, Israel

Acceptance Rates

IUI '14 Paper Acceptance Rate 46 of 191 submissions, 24%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
1,896
Total Downloads

Downloads (Last 12 months)201
Downloads (Last 6 weeks)20

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yu Yxia BXie MLi ZWang X(2024)Dynamic Modeling for Reinforcement Learning with Random DelayArtificial Neural Networks and Machine Learning – ICANN 202410.1007/978-3-031-72341-4_26(381-396)Online publication date: 17-Sep-2024
https://doi.org/10.1007/978-3-031-72341-4_26
Agrawal SMitra P(2024)Deep Reinforcement Learning in Healthcare and Biomedical ResearchDeep Reinforcement Learning and Its Industrial Use Cases10.1002/9781394272587.ch9(179-205)Online publication date: 4-Oct-2024
https://doi.org/10.1002/9781394272587.ch9
Che NZhang TLi YYu FWang H(2023)RLSF: Multimodal Sleep Improvement Based Reinforcement LearningIEEE Access10.1109/ACCESS.2023.326609411(47712-47724)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3266094
Lindley SLu YShukla D(2023)The Experimentalist’s Guide to Machine Learning for Small Molecule DesignACS Applied Bio Materials10.1021/acsabm.3c00054Online publication date: 3-Aug-2023
https://doi.org/10.1021/acsabm.3c00054
Xie MXia BYu YWang XChang Y(2023)Addressing Delays in Reinforcement Learning via Delayed Adversarial Imitation LearningArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44213-1_23(271-282)Online publication date: 22-Sep-2023
https://doi.org/10.1007/978-3-031-44213-1_23
Putze FPutze SSagehorn MMicek CSolovey E(2022)Understanding HCI Practices and Challenges of Experiment Reporting with Brain Signals: Towards Reproducibility and ReuseACM Transactions on Computer-Human Interaction10.1145/349055429:4(1-43)Online publication date: 31-Mar-2022
https://dl.acm.org/doi/10.1145/3490554
Ho JWang CChuang CKing CFeng CChou TChen YYang YHsiao Y(2022)Bootstrapping Human-Autonomy Collaborations by using Brain-Computer Interface of SSVEP for Multi-Agent Deep Reinforcement Learning2022 IEEE 3rd International Conference on Human-Machine Systems (ICHMS)10.1109/ICHMS56717.2022.9980765(1-6)Online publication date: 17-Nov-2022
https://doi.org/10.1109/ICHMS56717.2022.9980765
Kohli VTripathi UChamola VRout BKanhere S(2022)A review on Virtual Reality and Augmented Reality use-cases of Brain Computer Interface based applications for smart citiesMicroprocessors & Microsystems10.1016/j.micpro.2021.10439288:COnline publication date: 1-Feb-2022
https://dl.acm.org/doi/10.1016/j.micpro.2021.104392
Dumitrescu CCostea ISemenescu A(2021)Using Brain-Computer Interface to Control a Virtual Drone Using Non-Invasive Motor Imagery and Machine LearningApplied Sciences10.3390/app11241187611:24(11876)Online publication date: 14-Dec-2021
https://doi.org/10.3390/app112411876
Ji ZLiu QXu WYao BLiu JZhou Z(2021)A closed-loop brain-computer interface with augmented reality feedback for industrial human-robot collaborationThe International Journal of Advanced Manufacturing Technology10.1007/s00170-021-07937-z124:9(3083-3098)Online publication date: 10-Sep-2021
https://doi.org/10.1007/s00170-021-07937-z
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten