Nothing Special   »   [go: up one dir, main page]
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Neuronal ensemble control of prosthetic devices by a human with tetraplegia

Nature volume 442, pages 164–171 (2006)Cite this article

Abstract

Neuromotor prostheses (NMPs) aim to replace or restore lost motor functions in paralysed humans by routeing movement-related signals from the brain, around damaged parts of the nervous system, to external effectors. To translate preclinical results from intact animals to a clinically useful NMP, movement signals must persist in cortex after spinal cord injury and be engaged by movement intent when sensory inputs and limb movement are long absent. Furthermore, NMPs would require that intention-driven neuronal activity be converted into a control signal that enables useful tasks. Here we show initial results for a tetraplegic human (MN) using a pilot NMP. Neuronal ensemble activity recorded through a 96-microelectrode array implanted in primary motor cortex demonstrated that intended hand motion modulates cortical spiking patterns three years after spinal cord injury. Decoders were created, providing a ‘neural cursor’ with which MN opened simulated e-mail and operated devices such as a television, even while conversing. Furthermore, MN used neural control to open and close a prosthetic hand, and perform rudimentary actions with a multi-jointed robotic arm. These early results suggest that NMPs based upon intracortical neuronal ensemble spiking activity could provide a valuable new neurotechnology to restore independence for humans with paralysis.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Intracortical sensor and placement, participant 1.
Figure 2: Electrical recordings from a sample of four electrodes.
Figure 3: Neuronal selectivity for imagined and performed movements.
Figure 4: Directional tuning during centre-out task.
Figure 5: Reconstruction of neural cursor position during pursuit tracking.
Figure 6: Centre-out task performance.

Similar content being viewed by others

References

  1. Jackson, A. B., Dijkers, M., Devivo, M. J. & Poczatek, R. B. A demographic profile of new traumatic spinal cord injuries: change and stability over 30 years. Arch. Phys. Med. Rehabil. 85, 1740–1748 (2004)

    Article  Google Scholar 

  2. Thoumie, P. et al. Clinical and functional evaluation of a gaze controlled system for the severely handicapped. Spinal Cord 36, 104–109 (1998)

    Article  CAS  Google Scholar 

  3. Humphrey, D. R., Schmidt, E. M. & Thompson, W. D. Predicting measures of motor performance from multiple cortical spike trains. Science 170, 758–762 (1970)

    Article  ADS  CAS  Google Scholar 

  4. Schwartz, A. B., Taylor, D. M. & Tillery, S. I. Extraction algorithms for cortical control of arm prosthetics. Curr. Opin. Neurobiol. 11, 701–707 (2001)

    Article  CAS  Google Scholar 

  5. Paninski, L., Fellows, M. R., Hatsopoulos, N. G. & Donoghue, J. P. Spatiotemporal tuning of motor cortical neurons for hand position and velocity. J. Neurophysiol. 91, 515–532 (2004)

    Article  Google Scholar 

  6. Maynard, E. M., Nordhausen, C. T. & Normann, R. A. The Utah intracortical electrode array: a recording structure for potential brain-computer interfaces. Electroencephalogr. Clin. Neurophysiol. 102, 228–239 (1997)

    Article  CAS  Google Scholar 

  7. Guillory, K. S. & Normann, R. A. A 100-channel system for real time detection and storage of extracellular spike waveforms. J. Neurosci. Methods 91, 21–29 (1999)

    Article  CAS  Google Scholar 

  8. Suner, S., Fellows, M. R., Vargas-Irwin, C., Nakata, K. & Donoghue, J. P. Reliability of signals from chronically implanted, silicon-based electrode array in non-human primate primary motor cortex. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 524–541 (2005)

    Article  Google Scholar 

  9. Maynard, F. M. Jr et al. International standards for neurological and functional classification of spinal cord injury. American Spinal Injury Association. Spinal Cord 35, 266–274 (1997)

    Article  Google Scholar 

  10. Yousry, T. A. et al. Localization of the motor hand area to a knob on the precentral gyrus. A new landmark. Brain 120, 141–157 (1997)

    Article  Google Scholar 

  11. Lotze, M., Laubis-Herrmann, U., Topka, H., Erb, M. & Grodd, W. Reorganization in the primary motor cortex after spinal cord injury—A functional magnetic resonance (fMRI) study. Restor. Neurol. Neurosci. 14, 183–187 (1999)

    CAS  PubMed  Google Scholar 

  12. Shoham, S., Halgren, E., Maynard, E. M. & Normann, R. A. Motor-cortical activity in tetraplegics. Nature 413, 793 (2001)

    Article  ADS  CAS  Google Scholar 

  13. Sabbah, P. et al. Sensorimotor cortical activity in patients with complete spinal cord injury: a functional magnetic resonance imaging study. J. Neurotrauma 19, 53–60 (2002)

    Article  CAS  Google Scholar 

  14. Mikulis, D. J. et al. Adaptation in the motor cortex following cervical spinal cord injury. Neurology 58, 794–801 (2002)

    Article  CAS  Google Scholar 

  15. Donoghue, J. P., Sanes, J. N., Hatsopoulos, N. G. & Gaal, G. Neural discharge and local field potential oscillations in primate motor cortex during voluntary movements. J. Neurophysiol. 79, 159–173 (1998)

    Article  CAS  Google Scholar 

  16. Sanes, J. N. & Donoghue, J. P. Plasticity and primary motor cortex. Annu. Rev. Neurosci. 23, 393–415 (2000)

    Article  CAS  Google Scholar 

  17. Serruya, M. D., Hatsopoulos, N. G., Paninski, L., Fellows, M. R. & Donoghue, J. P. Instant neural control of a movement signal. Nature 416, 141–142 (2002)

    Article  ADS  CAS  Google Scholar 

  18. Taylor, D. M., Tillery, S. I. & Schwartz, A. B. Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 (2002)

    Article  ADS  CAS  Google Scholar 

  19. Evarts, E. V. in Handbook of Physiology (ed. Brooks, V.) 1083–1120 (Williams and Wilkins, Baltimore, 1981)

    Google Scholar 

  20. Georgopoulos, A. P., Kalaska, J. F., Caminiti, R. & Massey, J. T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982)

    Article  CAS  Google Scholar 

  21. Humphrey, D. R. & Tanji, J. in Motor Control: Concepts and Issues (eds Humphrey, D. R. & Freund, H. J.) 413–443 (John Wiley, London, 1991)

    Google Scholar 

  22. Wu, W. et al. Modeling and decoding motor cortical activity using a switching Kalman filter. IEEE Trans. Biomed. Eng. 51, 933–942 (2004)

    Article  Google Scholar 

  23. Carmena, J. M. et al. Learning to control a brain-machine interface for reaching and grasping by primates. PLoS Biol. 1, E42 (2003)

    Article  Google Scholar 

  24. Baker, J. T., Donoghue, J. P. & Sanes, J. N. Gaze direction modulates finger movement activation patterns in human cerebral cortex. J. Neurosci. 19, 10044–10052 (1999)

    Article  CAS  Google Scholar 

  25. Olson, B. P., Si, J., Hu, J. & He, J. Closed-loop cortical control of direction using support vector machines. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 72–80 (2005)

    Article  Google Scholar 

  26. Wolpaw, J. R. & McFarland, D. J. Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proc. Natl Acad. Sci. USA 101, 17849–17854 (2004)

    Article  ADS  CAS  Google Scholar 

  27. Birbaumer, N. et al. A spelling device for the paralysed. Nature 398, 297–298 (1999)

    Article  ADS  CAS  Google Scholar 

  28. Fetz, E. E. & Baker, M. A. Operantly conditioned patterns on precentral unit activity and correlated responses in adjacent cells and contralateral muscles. J. Neurophysiol. 36, 179–204 (1973)

    Article  CAS  Google Scholar 

  29. Fetz, E. E. & Finocchio, D. V. Operant conditioning of specific patterns of neural and muscular activity. Science 174, 431–435 (1971)

    Article  ADS  CAS  Google Scholar 

  30. Jacobs, K. M. & Donoghue, J. P. Reshaping the cortical motor map by unmasking latent intracortical connections. Science 251, 944–947 (1991)

    Article  ADS  CAS  Google Scholar 

  31. Gandolfo, F., Li, C., Benda, B. J., Schioppa, C. P. & Bizzi, E. Cortical correlates of learning in monkeys adapting to a new dynamical environment. Proc. Natl Acad. Sci. USA 97, 2259–2263 (2000)

    Article  ADS  CAS  Google Scholar 

  32. Schwartz, A. B. Cortical neural prosthetics. Annu. Rev. Neurosci. 27, 487–507 (2004)

    Article  CAS  Google Scholar 

  33. Hochberg, L. R. & Donoghue, J. P. Sensors for brain-computer interfaces. IEEE Eng. Med. Biol. (in the press)

  34. Kennedy, P. R. & Bakay, R. A. Restoration of neural output from a paralyzed patient by a direct brain connection. Neuroreport 9, 1707–1711 (1998)

    Article  CAS  Google Scholar 

  35. Kennedy, P. R., Bakay, R. A., Moore, M. M., Adams, K. & Goldwaithe, J. Direct control of a computer from the human central nervous system. IEEE Trans. Rehabil. Eng. 8, 198–202 (2000)

    Article  CAS  Google Scholar 

  36. Kennedy, P. R., Kirby, M. T., Moore, M. M., King, B. & Mallory, A. Computer control using human intracortical local field potentials. IEEE Trans. Neural Syst. Rehabil. Eng. 12, 339–344 (2004)

    Article  Google Scholar 

  37. Mason, S. G., Bohringer, R., Borisoff, J. F. & Birch, G. E. Real-time control of a video game with a direct brain–computer interface. J. Clin. Neurophysiol. 21, 404–408 (2004)

    Article  Google Scholar 

  38. Vaughan, T. M. et al. Brain-computer interface technology: a review of the Second International Meeting. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 94–109 (2003)

    Article  Google Scholar 

  39. Muller-Putz, G. R., Scherer, R., Pfurtscheller, G. & Rupp, R. EEG-based neuroprosthesis control: a step towards clinical practice. Neurosci. Lett. 382, 169–174 (2005)

    Article  Google Scholar 

  40. Kennedy, P. et al. Using human extra-cortical local field potentials to control a switch. J. Neural Eng. 1, 72–77 (2004)

    Article  ADS  Google Scholar 

  41. Leuthardt, E. C., Schalk, G., Wolpaw, J. W., Ojemann, J. G. & Moran, D. W. A brain-computer interface using electrocorticographic signals in humans. J. Neural Eng. 1, 63–71 (2004)

    Article  ADS  Google Scholar 

  42. Graimann, B., Huggins, J. E., Levine, S. P. & Pfurtscheller, G. Toward a direct brain interface based on human subdural recordings and wavelet-packet analysis. IEEE Trans. Biomed. Eng. 51, 954–962 (2004)

    Article  Google Scholar 

  43. Lal, T. N., et al. in Advances in Neural Information Processing Systems (eds Saul, L. K., Weiss, Y. & Bottou, L.) (MIT Press, Cambridge, Massachusetts, 2005)

    Google Scholar 

  44. Rousche, P. J. & Normann, R. A. A method for pneumatically inserting an array of penetrating electrodes into cortical tissue. Ann. Biomed. Eng. 20, 413–422 (1992)

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The authors thank J. Joseph and D. Morris for assistance; L. Mermel for clinical planning advice; V. Zerris and M. Park for surgical assistance; G. Polykoff for clinical trial assistance; W. Truccolo for power spectral density analysis development; and the employees of Cyberkinetics for device engineering, manufacturing and clinical trial design and management. The authors also thank MN for his participation in this trial, and the nursing staff at his assisted care facility for their assistance. The authors are grateful to M. Serra and Sargent Rehabilitation Center, the study site, for administrative support. The photograph of MN (Fig. 1) is copyright 2005 Rick Friedman. This work was supported by Cyberkinetics Neurotechnology Systems, Inc.

Author information

Author notes

  1. Maryam Saleh

    Present address: Graduate Program in Computational Neuroscience, University of Chicago, Chicago, Illinois, 60637, USA

Authors and Affiliations

  1. Department of Neurology, Massachusetts General Hospital, Brigham and Women's Hospital, and Spaulding Rehabilitation Hospital, Harvard Medical School, Boston, 55 Fruit Street, Massachusetts, 02114, USA

    Leigh R. Hochberg

  2. Department of Neuroscience and Brain Science Program,

    Leigh R. Hochberg, Mijail D. Serruya & John P. Donoghue

  3. Department of Engineering, Brown University, Providence, PO Box 1953, Rhode Island, 02912, USA

    Mijail D. Serruya

  4. Center for Restorative and Regenerative Medicine, Rehabilitation Research and Development Service, Department of Veterans Affairs, Veterans Health Administration, Providence, 830 Chalkstone Avenue, Rhode Island, 02908, USA

    Leigh R. Hochberg

  5. Department of Clinical Neurosciences (Neurosurgery), Brown University,

    Gerhard M. Friehs

  6. Department of Neurosurgery, Rhode Island Hospital, Providence, 120 Dudley Street, Suite 103, Rhode Island, 02905, USA

    Gerhard M. Friehs

  7. Department of Rehabilitation Medicine, Brown University, Providence, 593 Eddy Street, Rhode Island, 02903, USA

    Jon A. Mukand

  8. Sargent Rehabilitation Center, Warwick, 800 Quaker Lane, Rhode Island, 02818, USA

    Jon A. Mukand

  9. Cyberkinetics Neurotechnology Systems, Inc., Foxborough, 100 Foxborough Boulevard–Suite 240, Massachusetts, 02035, USA

    Maryam Saleh, Abraham H. Caplan & John P. Donoghue

  10. Cyberkinetics Neurotechnology Systems, Inc., Salt Lake City, 391 Chipeta Way, Suite G, Utah, 84108, USA

    Almut Branner

  11. Department of Physical Medicine and Rehabilitation, Rehabilitation Institute of Chicago, 345 E. Superior Street, 1146, Illinois, 60611, Chicago, USA

    David Chen

  12. Department of Neurosurgery, University of Chicago Hospitals, 5841 S. Maryland Avenue, MC3026, Illinois, 60637, Chicago, USA

    Richard D. Penn

Authors
  1. Leigh R. Hochberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mijail D. Serruya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerhard M. Friehs
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jon A. Mukand
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Maryam Saleh
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Abraham H. Caplan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Almut Branner
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. David Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Richard D. Penn
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. John P. Donoghue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John P. Donoghue.

Ethics declarations

Competing interests

L.R.H.: Clinical trial support, Cyberkinetics Neurotechnology Systems (CKI); G.M.F.: stock holdings, consultant, CKI; J.A.M.: principal investigator, consultant, CKI; M.D.S.: salary, consultant, stock holdings, CKI; M.S.: salary, stock options, CKI; A.H.C.: salary, stock options, stock holdings, CKI; A.B.: salary, stock options, CKI; D.C.: clinical trial support, CKI; R.D.P.: clinical trial support, CKI; J.P.D.: Chief Scientific Officer, compensation, stock holdings, director, CKI.

Supplementary information

Supplementary Notes

This file contains Supplementary Figures 1 and 2 and legends, Supplementary Methods and Results, Supplementary Discussion, Supplementary Video Legends and Supplementary References. The two figures illustrate neuronal selectivity for imagined and performed movements, and center-out task performance with an alternate post-hoc control. Also reported is additional information regarding signal quality and variety; MI activity during neural cursor control; center-out task; grid task; summary of neurophysiologic findings; comparison with previous work; video legends. (PDF 176 kb)

Supplementary Video 1

Center-Out task. (MOV 3821 kb)

Supplementary Video 2

Video showing use of a computer interface with the neural cursor. (MOV 1645 kb)

Supplementary Video 3

Neurally-controlled television. (MOV 1608 kb)

Supplementary Video 4

Neural "Pong". (MOV 2978 kb)

Supplementary Video 5

Neural "HeMan" game. (MOV 2888 kb)

Supplementary Video 6

Direct neural control of a prosthetic hand. MN was initially instructed to move a neural cursor "up" to open the hand, and "down" to close the hand. (MOV 2330 kb)

Supplementary Video 7

Transport of an object from one location to another via direct neural control of a multi-articulated robot arm. (MOV 1745 kb)

Supplementary Video 8

Trial Participant #2 performing Center-Out task. (MOV 7551 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hochberg, L., Serruya, M., Friehs, G. et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171 (2006). https://doi.org/10.1038/nature04970

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nature04970

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing