Abstract
In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.
Similar content being viewed by others
Opportunity and challenges
Neuroprosthetic devices have been recently proposed as promising solutions for restoring sensory-motor functions lost after injury or neurological disease, as described in Box 1. These devices extend implantable neural interfaces, which establish a functional connection pathway between the human nervous system (e.g., peripheral somatic nerves, cervical or lumbar spinal cord, or somatosensory cortex), and digital or robotic technology (e.g., computers, prostheses, or robotic devices). Similarly, electrical neurostimulation has been demonstrated to be a powerful tool for restoring sensory feedback in people with sensory loss (e.g., amputees or individuals with spinal cord injury). In contrast to purely motor neuroprostheses (such as those for restoring locomotion1, speech2 or hand functions3), sensory neuroprostheses require a bidirectional loop between the brain and the robotic device with both volitional control and sensing. Sensory neural interfaces require not only the ability to record, but also to deliver microstimulation selectively in order to activate the neural tissue; algorithms for controlling the multidimensional space of stimulation parameters in an effective and efficient manner; and wearable sensing technology for detecting body–environment interactions. However, restoring any sensory information is extremely complex due to its multidimensionality, both in time and space. Restoring somatosensory touch feedback requires sophisticated stimulation strategies that simultaneously modulate multiple parameters, e.g., active channels, pulse frequency, pulse amplitude, and pulse width. This results in restored sensory information which is still limited compared to the natural sense of touch. Multiple challenges are currently limiting the adoption and applications of somatosensory neuroprostheses in the clinical sphere. The main areas of current development include (1) neural interfacing (including material, biocompatibility, efficacy, etc.); (2) algorithms for decoding neural signals and encoding artificial sensations (e.g., real-time bidirectional systems, AI-based decoders, and biomimetic encoders); and (3) the engineering challenges of developing the hardware to perform and apply the necessary computations (e.g., wearable devices, fully implantable systems). Considering that presently there are electrodes implanted in humans, both in the PNS4,5 and CNS6 which have endured for many years, it is possible to develop and test novel algorithms for improving the efficacy and functionality of these devices. Recently, biomimetic neurostimulation that simulates the natural touch coding with in-silico models has been proposed as a promising approach to conveying sensory information in hand and leg prostheses and has been tested in first-in-human trials7,8,9,10,11. However, current state-of-the-art methods for obtaining realistic modeling simulations of the natural sensory processing, based on detailed biophysical descriptions, use computationally expensive processes implemented on external controllers which are connected to complex neurostimulators that cannot yet be embedded into implantable devices12,13,14. Translation of this approach into clinical practice would require implementing real-time algorithms that meet the hardware constraints of portable devices.
Neuromorphic technology provides alternative design solutions that present a paradigm shift where, rather than simulating the complex biophysics of mechanoreceptors, their behavior is faithfully emulated through the physics of electronic circuits. To achieve the restoration of the somatosensory experience, the neuromorphic system can translate sensory information, recorded by wearable sensors embedded in the prosthetic limb, and generate biomimetic neural stimulation patterns (Fig. 1). Contrary to traditional digital processors, this neuromorphic approach employs analog/digital mixed-signal CMOS-based hardware that supports in-memory computing and the implementation of neural computational primitives inspired by neural circuitry, such as biological time constants and adaptation at different time scales15. Neuromorphic computing has been extensively used for processing physiological data and generating actions, both in the domain of prosthetic control16,17 and bioelectronic medicine18,19. However, in the domain of somatosensory feedback restoration, neuromorphic hardware implementation has mainly focused thus far on the design of event-based sensors able to encode touch information in trains of spikes20, and/or in simulations of spiking neurons using traditional digital processors or field-programmable gate arrays (FPGA)21,22. In addition to using standard CMOS technology, there are several attempts at implementing neural interfaces using stretchable23, magnetometric24, and organic electronics25 with the goal of reproducing sensory feedback. Using soft electronics to build artificial skin brings several advantages in terms of tissue conformability, minimal invasiveness, and unobtrusiveness that increase the subject’s engagement26. However, to fully deploy this new technology, existing soft electronics require overcoming several challenges in biointegration, low power working regimes, and circuit complexity. Despite the current promising attempts (see Box 2 for more details), neuromorphic technologies still have a long way to go and could be further developed to increase the complexity of the modeling and include more mechanisms present in the afferent pathway, increase the degree of similarity with biological systems, and provide an approach that will revolutionize the field of neuroprosthetics. This new advancements will enable multiple benefits for users, including improved sensation naturalness, prosthesis embodiment, and home-use functionality. In addition, this approach will bring advantages in information encoding, providing an energy-efficient, compact, and embedded solution. However, there are still open challenges for this approach that range from the availability of a full neuromorphic pipeline (sensor–processor–stimulator) to the design of a proper interface for the generation of the actual pattern of stimulation, up to its translation in clinical trials.
The main building blocks of a neural prosthesis for the somatosensory system are the sensing block, the computing block, and the stimulating block. Sensing technology (e.g., wearable tactile sensors) has to be embedded in the robotic prostheses in order to extract all the relevant physical interactions with the external world. Then, the computing block, composed of the neuromorphic technology, has to translate the sensors’ readouts into electrical patterns of stimulation using biomimetic encoding algorithms. Finally, the stimulating block has to inject currents in the nervous tissue, through implantable electrodes, able to evoke natural neural activations, allowing for natural and informative sensations. A portion of the illustration is adapted from ref. 32.
In this perspective, we report existing works based on neuromorphic closed-loop systems, which are designed for sensing and controlling prosthetic devices through direct interfaces with the human nervous system in chronic and incurable neurological conditions27,28,29. We present the required steps to build the next sensory feedback for limb prostheeneration of neuromorphic hardware for restoring sensory feedback in neuroprostheses; as well as discuss the combined challenges of neuroprosthetic devices and neuromorphic technologies, suggesting possible research directions to overcome the current roadblocks and enable more natural human–machine interfacing.
Requirements for neuromorphic hardware for somatosensory neuroprosthesis
Combining advanced sensing systems with neuromorphic encoding schemes and novel neural interfaces would allow for highly functional sensory neuroprostheses, a higher sense of embodiment, and a more complete integration in the human body schema. A closed-loop neuromorphic prosthesis would also provide benefits in terms of information bandwidth, parallelization, and portability, allowing for improved biomimetic feedback.
Interfacing to the human nervous system
Neural interfaces for communicating with the brain through neurostimulation
The field of implantable neural interfaces has witnessed rapid progress in the past 30 years30,31,32. Indeed, several breakthroughs in the interfacing of the nervous system and electronics have been achieved, many of which have clinical relevance. When entering the phase of clinical application, it is of paramount importance to understand the working mechanisms of neural stimulation and to develop fully integrated neurotechnology able to effectively replace or repair as many biological functions as possible. This can substantially improve the quality of life of patients with sensory-motor deficits.
Different types of neural interfaces have been adopted to record the neural activity from the brain33,34,35 or nerves36,37,38 and, more recently, to also deliver electrical stimulation to peripheral nerves4,11,39,40,41,42,43,44 (see Box 3 for more details), spinal cord45,46, or cortex47,48,49 to restore sensory-motor functions. The use of electrical stimulation patterns allows for a valuable and reliable tool to directly communicate with the human nervous system enabling it to encode artificial sensory information50. Its effectiveness and functionality for sensory feedback restoration are strongly dependent on the characteristics of the adopted neural interface (i.e., implantable electrode). Indeed, the implanted electrode should be highly: (1) biocompatible; (2) selective; (3) modular; and (4) stable.
Biocompatibility is defined by the intrinsic properties of an electrode such as the materials it is made of, its size, and the implantation procedure. A biocompatible interface should be made with certified and standard materials for active implantable medical devices as required for clinical application51. Another important factor affecting biocompatibility is the difference in Young’s modulus between the electrode and the nervous tissue. The smaller the difference, the less the mechanical stress of the interface on the biological tissue52. Also, the size ratio between the electrode and the neural tissue could affect its biocompatibility. Smaller electrodes may have less impact on the biological environment potentially improving the integration of the device in the body, but at the expense of robustness and maximum injectable charge. Finally, the level of invasiveness of the implantation substantially influences the foreign body reaction (FBR) or inflammatory response that can be responsible for electrode failures or a decrease in their effectiveness53,54. Notably, recent results have demonstrated that non-invasive technologies using remapped evoked sensations are less informative, and intuitive than the neurally evoked somatotopic sensations restored via implantable devices55,56.
Electrode selectivity is currently of great interest to many research groups. Indeed, selective stimulation (i.e., defined as the capability of the electrode to activate small groups of neurons without unintentionally activating other neural regions) could allow substantially improved effectiveness of these interventions, minimizing the side effects. The selectivity is dependent on electrode size, shape, active site configuration, and stimulation protocol. In general, electrodes penetrating the nervous tissue (such as intraneural or intracortical) allow for more intimate contact of the electrically active sites with the target neurons. This scenario, together with a smaller active site area compared to extraneural or epicortical electrodes, guarantees a more focal stimulation, activating fewer neurons simultaneously. Moreover, electrodes with multiple active sites and also with multipolar configurations are able to shape the electric field achieving more selective stimulation57. Thanks to the high selectivity, the stimulation serves its intended purpose with reduced side effects. More recently, stimulation protocols co-modulating injected charge and high frequency could potentially increase the selectivity of the adopted interface and therefore its effectiveness58.
In addition, electrodes allowing for high maximum injectable charge and stimulation frequency, yield a wider space for parameter modulation. This is of crucial importance for developing more sophisticated stimulation strategies allowing the encoding of more complex features of the sensory experience. A modular interface exploits specific active site materials and coatings to improve injectable charge limits32,59.
The purpose of using implantable neural interfaces is to obtain an effective neural link that can be exploited for long-term applications in everyday life. To this aim, the interface should remain stable and reliable over time. Notably, the stability is strongly dependent on the electrode materials, shape, and implantation procedure. The invasiveness of the implant, in this case, decreases the potential stability of the electrode since the FBRs and the electrode migration are more likely to modify the electrode–neural tissue interface32,60. However, the majority of the implants adopting penetrating electrodes, in particular in Brain–Computer Interface (BCI) applications, lasted for several years in human trial applications (more than 10 years)6,61. Nevertheless, novel electrode materials and coatings have the potential to improve the long-term stability of intracortical and intraneural interfaces62.
Biomimicry as a solution for restoring naturalistic sensation
The human skin is an incredibly complex and sophisticated deformable organ capable of sensing any type of mechanical interaction of the human body with the environment. For instance, the skin on our hands is innervated by tens of thousands of mechano- and proprioceptive receptors, each of which carries different (albeit overlapping) information on items that are being gripped63. This neural information is then processed at multiple levels along the somatosensory axis from the dorsal root ganglion (DRG) to the cuneate nucleus, thalamus, and then to the primary somatosensory cortex64. From the idiosyncratic spiking responses of each of the afferents, the tactile information is processed and becomes a conscious percept. To artificially communicate with this complex system, a detailed knowledge of the neural geometry of touch is of crucial importance. Indeed, restoring natural touch through neural interfaces would require stimulating each of these sensory neurons independently with their own idiosyncratic activation patterns, in a way that current technologies are nowhere near ready to implement. State-of-the-art neural interfaces have tens or hundreds of stimulation channels31, not the thousand and more that would be required for full biomimetic restoration of touch on the palmar surface of the human hand. Moreover, each electrode activates tens or hundreds of axons and elicits highly unnatural synchronous activation of these afferents. Until much denser and more selective neural interfaces become available, attempts to mimic natural nerve responses will have to settle for mimicking aggregate neural responses14,50,65. In the natural sensory processing condition, the spatiotemporal dynamics of this aggregate population response in the sensory cortex seem to mirror those of the population response in the nerves66, making the assumptions more generalizable for the entire somatosensory neuroaxis. Moreover, considering that the natural sensory system is encoding an enormous amount of information about the tactile experience (such as intensity, type, location, temperature, and other characteristics of the tactile stimulus), one way to mitigate the complexity issue is to translate this information into the artificial sensory space exploiting biomimicry67. Indeed, this method exploits, as far as possible, existing perceptual representations of natural sensory processing. In fact, to the extent that we can reproduce the neuronal firing patterns that would occur with an intact nervous system, the resulting perceptions could be more natural9 and would also require minimal learning. While we cannot completely reproduce natural patterns of neuronal firing, we can exploit key principles of sensory processing in the development of the encoding algorithms7,8. In particular, the spatial and temporal components of the neural activation could be modulated and partially controlled through the neurostimulation parameters8,68. This would allow the exploitation of residual sensory processing for evoking natural and informative percepts.
Currently, in the neuroprosthetic field, the scheme of neural stimulation is mostly not defined by the nerve’s natural coding or neuromorphic models (Fig. 2A)50. Indeed, after defining the pulse waveform of the stimulation train, the modulation occurred only on a single parameter and mostly on individual channels of the neural interface. This unnatural approach causes the evoked sensations to be mostly described by users as vibration, tingling, paresthesia, or electricity32. In fact, paresthetic sensations are likely to arise from the unnatural activation of nerve fibers58 and can be due to the over-excitation of afferents or cross-talk between them69. The encoding functions that are used to modulate the injected charge (pulse width or pulse amplitude)4,39,40 or pulse frequency70,71 of the delivered neurostimulation pulse–train according to values read from the prosthetic sensor. This mapping generally follows a linear and proportional relationship to the sensor readings (e.g., the higher the value of the pressure measured by the sensor is, the higher the stimulation charge will be). Therefore, the intensity of the perceived sensation is proportionally associated with the stimulation charge and frequency (the higher the injected charge is, the higher the perceived sensation intensity will be)47,72. Interestingly, the use of sinusoidal modulation of the stimulation pulse width has shown promising results in improving sensation quality40. Unfortunately, these findings have not yet been replicated by other research groups73. The sensory transformation from paresthesia to natural qualia seems to require more than patterned and sinusoidal neurostimulation.
A The whole pipeline of neural stimulation patterns for somatosensory feedback restoration based on non-biomimetic design and B based on biomimetic design. In the latter case, the stimulation parameters, such as frequency, amplitude, and stimulating channels are modulated in order to evoke natural patterns of neural activation. A portion of the illustration is adapted from ref. 32.
In fact, natural touch coding, and the relationship between biological sensors (cutaneous receptors) and neural activity is more complex than a sole single-parameter coding. For example in natural touch, the information about transients is much more salient and prominent in the neural code than the information about sustained stimulation63,66, as it includes both rate and temporal coding. Thus, a more biomimetic pattern should incorporate this information, highlighting the contacts7, and should modulate multiple parameters and electrodes at the same time7,9. The target aim is to electrically induce a natural pattern of neural activation67, such as the one generated by the biological receptors in the case of healthy touch systems. The theory of adopting more biomimetic and bio-inspired patterns of stimulation assumes that replicating the natural firing patterns would lead to more natural and intuitive sensations (Fig. 2B). The spatio-temporal modulation (multi-parameter and multi-channel) of the neural stimulation would allow to generate more complex and natural evoked activity of the recruited neuron population. To do so, computational models12,13,14, able to replicate the natural neural responses, could be adopted. To develop such models is fundamental to have ground truth of the natural spatiotemporal activation that the artificial system needs to replicate. This will require models able to run in real-time and able to generalize to every type of tactile input coming from the external environment. Neuromorphic technology can provide systems able to emulate (with low power consumption) the natural neural responses of different neurons/networks and also to learn and adapt to different tactile inputs.
Neuromorphic technology in neuroprosthetics
Neuromorphic hardware to boost neuroprosthetic applications
Neuromorphic technologies use event-driven, parallel, and in-memory computing techniques to emulate computations performed by biological nervous systems74,75. They comprise of neuron and synapse circuits implemented using standard CMOS technologies with biologically plausible time constants (from milliseconds to seconds or even minutes) that make them ideal candidates to emulate neural dynamics76. Unlike conventional digital technologies, neuromorphic processors implement memory storage co-distributed with the processing, which allows memory and power consumption to be minimized77. Moreover, they often support network reconfigurability17,78, conveying information as temporal spikes rather than numeric values79. Spikes allow sparse and asynchronous communication which relies on the time of firing and enables efficient spatiotemporal pattern discrimination80. Box. 4 outlines the features of some representative neuromorphic processors.
In the field of neuroprostheses, neuromorphic solutions are increasingly explored to build tactile sensors and restore feedback sensation. In the past decade, different ways to create an artificial sense of touch have been explored using different sensors, e.g., resistive81 piezoresistive82,83, capacitive84, optical85, piezoelectric86,87, or acoustic88. As in biology, event-driven sensors locally encode changes of stimulus intensity with a train of spikes utilizing a specific dynamic temporal pattern, in contrast to conventional tactile sensors that are sampled periodically and return a continuous scalar pressure value even when no force is applied. Encoding changes decrease the data to process, allowing a more energy-efficient and biomimetic approach. Current implementations of neuromorphic tactile sensors can be divided into two classes: soft neuromorphic, where tactile sensors are combined with a microcontroller or other digital processors to simulate spiking model neurons, or neuromorphic sensors that directly output spike trains.
In soft applications, the sensor outputs are transmitted into the digital processor or FPGA where a neuron spiking model encodes the signal into spikes21,22. Although these implementations are convenient solutions for system-level applications, the need for a digital device to simulate simple spiking neuron models limits the power performance and miniaturization. This approach allows the implementation of multisensory systems with high spatial resolution. An example includes a 64 × 64 resistive sensor matrix combined with a piezoresistive fabric with a sub-millisecond response time and a sampling frequency of 5 kHz. The sensory array is connected to an FPGA to handle the parallelism of the sensors and to generate spikes every time a significant change is detected for a particular element89,90. Another implementation is composed of two by two microelectronic mechanical systems (MEMS) arrays, where each sensor is composed of 4 piezoresistors working at 375 Hz interfaced with an FPGA that simulates a simple model of spiking neurons, the Izhikevich model91, to reproduce the behavior of tactile receptors22 which can be applied to the categorization of naturalistic textures92. The asynchronously coded electronic skin (ACES) consists of 240 artificial mechanoreceptors that asynchronously transmit spikes with a latency of 1 ms and an ultra-high temporal resolution precision of <60 ns. Each receptor consists of a resistive sensor, a microcontroller for the generation of the spike, and a series of passive components to perform signal conditioning93. Soft implementations are used for both stimulations, mainly non-inasively21,94,95, and, more extensively, for texture discrimination. The feasibility of texture discrimination has been demonstrated using different approaches, with the most notable approach using piezoelectric sensor arrays and the Izhikevich spiking neural model. Various analytical methods were applied for the investigation of spatiotemporal spike patterns, such as spike train distance-based kNN96, interspike interval statistics and spike distance metrics92, a recurrent spiking neural network97, and an extreme learning machine98. Another approach is based on 24 capacitive square sensors in a rectangular grid layout working at 20 kHz. The output is proportional to the intensity of the force and it is converted into spikes using an integrate and fire neuron utilizing an adaptive threshold. This has been used for the classification of Braille stimuli99. However, while these implementations can serve as tools for the development of applications, they do not reach the same low-power performance and miniaturization of directly designing biomimetic sensors, and the neural model investigated is rather simple, although other neuron models can be explored.
Current neuromorphic implementations based on mixed-mode subthreshold circuits represent promising solutions to solve the problem of power consumption and miniaturization. However, there is the need for multiple transducers to emulate different mechanoreceptors, and the difficulty of their mechanical interfacing limits their stable integration on prostheses or robotic platforms100. They include piezoelectric transducers99,101,102, which detect fast changes, and hence emulate the behavior of rapid adapting mechanoreceptors (RAs), and capacitive transducers which have the advantage of keeping their capacitance over time enabling the emulation of slow adaptive mechanoreceptors (SAs)103. Another example uses semi-volatile carbon nanotube transistors to build a sensor system with sensory neurons and a perceptual synaptic network to differentiate the temporal features of tactile patterns104. In both implementations, the silicon neuron is a LIF neuron with the basic neural function of integrating the incoming information and encoding the information into a spike once the membrane voltage reaches a threshold.
At present, neuromorphic implementations are still in a proof-of-concept stage, and they have not yet been tested in neuroprosthetic applications. However, they have been deployed for spatiotemporal discrimination tasks, such as Braille letter reading105. Very recently, Wang and colleagues presented a promising monolithic soft prosthetic e-skin capable of multimodal perception, neuromorphic pulse-train signal generation, and closed-loop actuation. With a trilayer, high-permittivity elastomeric dielectric, they achieved a low subthreshold swing comparable to that of polycrystalline silicon transistors which operate at a low voltage, consume little power, and integrate medium-scale circuits that enable stretchable organic devices26.
A third explorable alternative is the integration of standard sensor arrays with neuromorphic technologies, where the mechanoreceptors’ behavior is not simulated in a digital processor but emulated in silicon neurons, and their behavior is set by the neurons’ parameters. This approach replaces the digital processor of the soft solution with a more power-efficient solution; however, it adds more circuit complexity than that present in event-based pixels. The adaptation in the mechanoreceptors can be emulated by a non-linear adaptive neuron circuit with a calcium channel responsible for the spike frequency adaptation in real neurons106. The silicon neuron model can vary from a complex conductance-based Hodgkin–Huxley model with a high degree of biological compatibility107, to the Adaptive Leaky Integrate-and-Fire (AD-LIF) model. Recent implementations of AD-LIF silicon neurons use 22 nm technology, with a layout area of 15 × 60 μm2 per neuron, an adaptation time constant of up to 5 s, and a power consumption of 14 pJ/spike, that enables its use in edge computing applications, including integration in neurorobotic applications108.
Compared to standard approaches, neuromorphic tactile sensors offer a solution to the drawback of limited bandwidth, both in terms of the frequency of the acquired data and the high-density receptor arrays, by using populations of spiking neurons with limited bandwidth (e.g., able to fire at most at a few Hertz or tens of Hertz) that can collectively encode signals with much higher bandwidth and dynamic range109. Sensors could transmit the generated spikes by using the address event representation (AER) communication protocol, which is the most common protocol in neuromorphic technologies, and utilizes the address of the source or destination neuron to distinguish the spikes that travel on a shared digital bus110. To further improve the distributed sensory transmission, a compressive method on the sensor side using asynchronous impulse radio ultra-wideband (IR-UWB) for wireless low-power communication can be explored111. Wireless communication using IR-UWB could improve system acceptance and usability reducing the use of wires that pass along the prosthetic device.
Pattern generation for electric stimulation
Once appropriate mechanoreceptor output spike trains are generated, they serve as the input to generate the activation pattern for electric stimulation. Generating an effective natural pattern requires a compression stage and a transformation in the stimulator’s parameters, combining multiple spike trains into a single train for use in clinical applications. Common approaches include computing the sum (representing an aggregate neural response) or the average of the output spike trains9, which may result in a high firing rate where spatial information could be compromised. In traditional machine learning, the use of compression algorithms, such as clustering and classification, are utilized to extract important features in real-time applications112. In cortical recordings, multiple spike trains can be merged into a single spike train by identifying temporal spike patterns within mechanoreceptors’ spike trains by analyzing temporal correlations with statistical methods such as entropy, or cross-correlation113. In neuromorphic sensors, while some studies address temporal compression114, only a few have focused on spatial and temporal down-sampling on event-based data, primarily relying on hardware filters and focusing on event-based cameras115,116.
To date, only soft neuromorphic solutions have been interfaced with non-invasive stimulation through transcutaneous electrical nerve stimulation (TENS)94,95, where the number of spikes is calculated at each time point and a threshold is set to define the sensitivity and accuracy of the spiking behavior21.
An unexplored approach is the use of a recurrent neural network that receives the mechanoreceptors’ output as its input and analyzes sequences by storing information in its internal memory state and outputs a singular pattern117. Although the network benefits from potential implementation on a neuromorphic chip, its implementation is not straightforward since it would require an appropriate training function, similar to autoencoders, where the compressed information is used to recreate the original signal. In addition, to overcome the problem of misclassification or regression error, we can add a safety mechanism at the output of the network to check the range of the rate of stimulation, avoiding the generation of an unpleasant pattern of stimulation. This safety mechanism can be implemented by an additional layer of recurrent inhibition that compresses the firing rate in a signal with a high dynamic range118 or by exploring a new approach of using hierarchical reservoir networks which allows networks to function across markedly diverse timescales, exceeding the performance achievable by single reservoirs119.
The compressed spike train is then used as input for the electric stimulator where the spikes are converted into pulses in real time. This conversion can introduce some drawbacks such as delays from the sensor’s output to the electrical stimulation120, or a lack of dynamics in the stimulation’s parameters (e.g., frequency, pulse width, etc.)95,121,122. To preserve the temporal information carried in the spikes and enable real-time operation, the conversion of the signal to spikes is critical21. Three strategies have been proposed and are based on prior work that shows biological significance under electrical stimulation for eliciting sensory perception:72,123 pulse width modulation, with fixed frequency and the pulse width dependent on the number of spikes per window; pulse frequency modulation with fixed pulses and frequencies proportional to the number of spikes counted in a time-programmable window; and a neuromorphic match where the temporal information is kept in the neuron’s spiking activity. In the latter approach, the neuron model encodes the information in frequency output, resembling natural modulation. Using silicon neurons rather than simulated neurons will allow for the use of the output neurons as pulse generators by adding a pulse extender circuit to the output of the neuron124. The pulse extender enables the generation of pulses with settable width, maintaining compact and low power implementation.
So far, the clinical implementations of neuromorphic approaches are limited to soft neuromorphic sensors in non-invasive stimulation, leaving room to explore neuromorphic solutions in neurostimulation for both sensors and processors.
Neuromorphic neurostimulator
Invasive closed-loop applications are still based on biomimetic approaches and, currently, neuromorphic approaches have yet to be fully explored. Biomimetic approaches involve the simulation of mechanoreceptors using powerful simulators running on digital processors12,13,125; however, when the complexity of the information encoding increases, it becomes computationally expensive and requires bulky stimulators, hindering real-time portable implementations. As described above, neuromorphic technologies promise a solution by using simulators to emulate the mechanoreceptors’ behavior thanks to their intrinsic properties, thus allowing portable solutions. Different mechanoreceptors can be emulated by setting different parameters in the chip, i.e., threshold, and adaptation time constants126 to match both slow and rapidly adapting mechanoreceptors (SAs and RAs). Skin deformation can be emulated by implementing mesh networks, in which neurons are connected with their neighbors via weights that depend on the emulated mechanoreceptor type. The higher the contact force, the further the propagation from the contact point.
To design an end-to-end neuromorphic pipeline and exploit all the advantages of the neuromorphic paradigm, the electric stimulator must be seamlessly incorporated. CMOS-based electric stimulators have been recently proposed and are regarded as ideal candidates127,128. Peripheral neural stimulation requires biphasic current pulses (from 10 to 1000 µA) to avoid charge accumulation, at the tissue interface, which would otherwise severely damage neighboring tissue. To design efficient peripheral stimulators, in terms of the amplitude resolution of the stimulus, current-mode stimulators are often preferred, although voltage-mode stimulators are very efficient in terms of power consumption129. In current-mode stimulation, the amount of delivered charge is set by controllable parameters such as the injected current and the duration of the pulse128. The major issue with current-mode stimulation is the high variability of electrode-tissue impedance (from 10 kΩ to 1 MΩ) that can generate a drop in the voltage. To overcome this limitation, the system can operate under a high voltage supply (from 15 V up to 30 V) that can be generated, and controlled, by embedding programmable voltage boosters. In addition, CMOS stimulators can be affected by mismatches that modify the generated pulses. This can be mitigated by different techniques such as blocking capacitors, or by using resistors connected to the terminals130. By using the impulse radio ultra-wideband (IR-UWB) the stimulator can be implanted close to the targeted nerve and can receive input from the processing unit, enhancing the embeddability of the system and simultaneously enhancing the comfort and acceptability for the subject131.
Location of electric stimulation along the ascending pathway
In addition to the many challenges described in tactile feedback restoration, there remains a need to determine the suitable location for electrical stimulation which depends on the severity of the neural injury and the degree of residual function. Stimulation can target either the peripheral or central nervous systems, but with fundamental differences in the information processing, and limitations imposed by the anatomy and physiology of the site132. Peripheral stimulation includes stimulation of primary afferent neurons in the residual limb39,133 and of the DRG134. Targeting primary afferents and the DRG has the advantage of utilizing the distinct behavior of the various receptor types to directly generate complex stimulation patterns. However, since the somatotopy is not guaranteed within nerves, the specific location of the different fibers is unknown a-priori. Thus, very selective interfaces and a long phase of mapping are required for effective targeting of the PNS. Central nervous system stimulation includes stimulation of the spinal cord45,135,136, the ventral posterolateral thalamus137, and the somatosensory cortex47,138. Stimulation at the spinal cord level presents some advantages compared to stimulation in the residual limb, such as more discernable somatotopy, valid for both distal and proximal amputation; as well as more robust implantation procedures. However, the low selectivity of the spinal electrodes affects the spatial resolution of the restored feedback. Finally, direct stimulation of the somatosensory cortex (Brodmann’s Area 1) using intracortical electrodes (ICMS) exploits the somatotopic organization of the sensory representation, and induces less paresthetic sensations when compared to peripheral stimulation. However, since the neurons are organized in networks and columns, ICMS evokes both direct and indirect neural activation, even in nearby areas (e.g., motor cortex—M1). This can complicate the design of closed-loop BCI10. Notably, the higher the stimulation location in the somatosensory neuroaxis, the more the mechanoreceptors’ outputs would require transformation; specifically firing depression and signal integration. Therefore, additional neural and synaptic mechanisms, such as spike frequency, short-time depression, and excitatory–inhibitory balance are required. Due to the high flexibility of their parameter and network reconfigurability, neuromorphic technologies could allow for stimulation at any level within the pathway without the need to rethink and redesign the stimulator.
Biomimetic neuro-robotic devices
The implementation of a natural neurostimulation approach for sensory restoration requires sensitized neuro-robotic prostheses. A prosthetic device will be used in daily life where the user has to accomplish a variety of motor tasks. The natural feedback provided to the user should match the body–environment interactions in terms of (1) time (i.e., real-time feedback; the sensation has to be perceived without delay with the physical interaction); (2) space (i.e. somatotopic feedback; the sensation has to be perceived in the same location of the physical interaction); (3) modality (i.e., homologous feedback; the type of perceived sensation has to match the type of the physical interaction); (4) intensity (i.e., modulated feedback; the intensity of the sensation has to match the intensity of the sensory experience); (5) naturalness (i.e., natural feedback; the evoked sensation has to be almost identical to a sensation experience on an intact hand). When an artificial sensory modality fulfills all these requirements, it can be considered comparable to a natural sensory modality and will provide high prosthesis embodiment and integration139.
To fulfill this specific design, the neuro-robotic prosthesis should be equipped with multiple wearable sensors able to detect robustly and reliably, in real-time, the interactions with the external world. Signals from multiple sensors have to be streamed and integrated, obtaining a reliable and stable input for the sensory encoding. The neuroprosthesis would require a robotic prosthesis equipped with wearable sensors, a neuromorphic controller chip, and a stimulating system. Sensor information is transmitted to the chip, which transduces them (using the biomimetic encodings, i.e., transfer functions based on natural sensory processing) into instructions for the neural stimulator. In particular, the neuromorphic hardware, embedded in the prosthetic device, would convert the artificial readouts of the sensors into biomimetic neurostimulation patterns. The translated signals are then converted into impulses of current, which are delivered to the residual peripheral nerve through electrodes, implanted directly into the nerve itself. This must be performed in real-time, with latency below 100 ms, to elicit an unperceivable delay for users140. This architecture will constitute a neuromorphic sensory neuroprosthesis. At the core of the closed-loop biomimetic prosthesis lies the ability to seamlessly capture, process, and transmit tactile/pressure information. Neuromorphic hardware can process in real-time such information and, by mimicking the intricate neural encoding mechanisms of the somatosensory system, can enable artificial limbs to provide users with natural and informative feedback (Fig. 3).
The complete design of a closed-loop biomimetic neuro-robotic prosthesis exploiting neuromorphic hardware is depicted. The pressure events under the prosthetic foot are sensed by the wearable sensors embedded in artificial skin. The stream of information from these sensors is the input for the neuromorphic chip that converts them into bio-inspired neural stimulation patterns resembling natural somatosensory processing. The co-modulations of the neurostimulation parameters and the channels of the implanted neural interfaces will allow the evoking of natural patterns of activation in the residual nervous system of the user. The electrically evoked sensation will be natural and informative allowing for the maximal exploitation of the robotic prosthesis in the sensory-motor loop. A portion of the illustration is adapted from ref. 42. Grafica_001 © 2021 by Pietro Comaschi is licensed under CC BY 4.0.
Then, the intact nervous system does the rest: the signals from the residual nerves are conveyed to the brain of the user, who is able to perceive what happens at the prosthesis and adjust the motor behavior accordingly. The machine and the body are finally connected.
Future directions and open questions
Humans possess fine motor skills that aid in accurately using and manipulating objects and tools, maintaining balance and walking, or performing any number of highly complex daily activities. In patients with amputations, missing functions can be restored using prostheses. However, dexterous prostheses are highly sophisticated and require the restoration of sensory information to improve the quality of control and acceptability of the patients, enabling a more natural and pleasant experience with the external device.
Analyzing the current state of the field, some recent works showed the first evidence of how biomimetic neural stimulation could be effectively adopted to encode more natural and functional somatosensory feedback in neuroprosthetics. Indeed in PNS, biomimetic stimulation (both for upper- and lower-limb) showed to evoke more natural precepts8,9 and more informativeness feedback to accomplish functional tasks8,9,11. In CNS, multichannel biomimetic ICMS conveys finely graded force feedback in BCI that more closely approximates the sensitivity conferred by natural touch7.
Regarding the engineering challenges, neuromorphic technologies present properties that enable the functional and structural replication of the nervous system, allowing for an ideal solution that integrates with existing neural interfaces, or for the creation of new interfaces in neurorobotic applications.
To make advances in the design of neuromorphic hardware for sensory restoration, and even more in general for the biomimetic approach, it is essential to develop more sophisticated event-based sensors where the sensors are integrated with a silicon model able to generate complex dynamics to resemble biological mechanoreceptors in a compact and low-power design, allows for powerful portable solutions. It is also important to understand how to encode multiple spike trains generated into fewer trains which can then be used to instruct the electric stimulation. This requirement is a consequence of the limited selectivity of the actual sensors, which can be overcome in the future by improving the electrode selectivity and signal bandwidth, more precisely targeting the desired nerve, hence, enhancing discrimination of the elicited sensation.
In addition to the required selectivity, interface stability is a crucial factor for the development of long-term solutions. The implantation of electrodes in human peripheral nerves is associated with biological responses in the nerve due to foreign body reactions (FBRs), causing changes to the properties of nerve–electrode interfaces. Similar FBR was observed for various penetrating electrodes (e.g. Utah array141 or polyimide-based electrodes60) implanted in human peripheral nerves. The encapsulation of the electrodes and their possible migration are among the theorized causes of poor stability over-time, in particular for evoked sensation location and perceptual threshold. The growth of fibrotic tissue could influence the effectiveness of the neurostimulation (e.g., lower conductivity due to a physical barrier between the electrode and neurons). Interestingly, pharmacological strategies have been proposed to modulate the FBR to neural implants142. The use of a collagen coating containing antifibrotic drugs on the intraneural electrode could enable a reduced growth of fibrotic tissue and macrophage infiltration around the implants. The increase in the biocompatibility (i.e., bio-integration and durability) of the device would guarantee the longer-term stability of the implant. Due to the problem of the electrodes’ possible failures and other biological reactions of the nerve (e.g., demyelination, changes in fiber distribution, etc.), constant re-calibration of the neural stimulation would be required. A possible solution to this issue is the exploitation of AI-based algorithms (e.g., Bayesian optimization, reinforcement learning, etc.) able to recalibrate the neurostimulation, identifying the best parameter configuration in a fully automatic and smart way143. This combined development of innovative hardware, materials, and software will allow for a future generation of neural interfaces able to provide effective neurostimulation for long-term applications.
Regarding the challenges in interfacing with the nerves, one of the barriers to the success of the approach presented here is related to the ability of evoking a desired spatiotemporal pattern of activation in the targeted neurons. Firstly, even with the latest and most selective available interfaces, the number of neurons activated with each pulse of stimulation is too high. In addition, all the neurons are simultaneously activated causing synchrony that is not present in vivo during natural touch67. Indeed, the natural asynchronous activation is driven in part by the probabilistic nature of action potential generation in sensory organs, such as muscle spindles144 or retinal cells145, and in part by the stochastic nature of synaptic transmission146. Recently, interesting approaches exploiting high-frequency electrical stimulation have been proposed to desynchronize neural activity58,147. However, the effect of high-frequency stimulation could induce undesired strong sensations, since frequency modulation has an effect on perceived sensation intensity148. Another option could be to exploit other techniques in activating the neural tissue, such as optogenetic, and ultrasound stimulation. Indeed, optogenetic stimulation potentially provides the ability to target molecularly defined neuron subtypes, access opsins engendering neural inhibition, and optically recruit axons in a fashion that might mimic natural recruitment149,150. In addition, the neuromorphic approach could also be adopted for other sensory prosthesis designs, in which the biomimetic approach has been proven to be effective for improving functional performance (e.g., enhanced speech intelligibility for cochlear implants;151 and improved restoration of gaze stability in vestibular prostheses152). Thus, the development of prostheses that can account for such pathway-specific heterogeneities will be essential to improving functional outcomes across sensory systems.
Regarding the specific considerations that need to be addressed when generating output signals for PNS stimulation, the safety limits of amplitude, pulse width, and frequency must always be considered to avoid electrode degradation, tissue damage, fast adaptation (Box 5), and sensations that are too strong. Notably, possible interferences with other sensory modalities such as proprioception or motor signals, due to undesired afferent activation, are currently under investigation by research groups working in the field of bidirectional sensory-motor prosthetics10.
Finally, the development of fully biomimetic neuroprostheses, able to restore natural and effective sensations, requires a quantitative assessment of the naturalness of this synthetic somatosensory feedback. This assessment represents a further challenge in the development of this type of neuroprosthesis since sensation naturalness is highly subjective. The lack of a bio-signal to objectively quantify the error between the evoked and target sensations makes the development of efficient algorithms very challenging. Perceived naturalness could be influenced by past or present experience, and could be modified with the modulation of stimulation parameters and its related perceived intensity153. To this aim, a detailed evaluation is needed, combining both subjective (e.g., questionnaires, free description, psychophysics) and objective measures (e.g., reaction times, neural recordings, performance metrics, learning).
Our perspective informs the design of a future generation of bio-inspired neuroprosthetic devices allowing the artificial conveyance of more complex sensory information with neurostimulation. The approach proposed here could potentially increase, not only the efficacy but also the acceptance of neuroprosthetic devices, improving the quality of life of people with sensory impairments. The use of neurostimulation is also critical to brain–computer interfacing, as well as bioelectronic medicine, in which the electrical stimulation targets the central, or the autonomic, nervous systems, respectively.
References
Wagner, F. B. et al. Targeted neurotechnology restores walking in humans with spinal cord injury. Nature 563, 65–71 (2018).
Metzger, S. L. et al. A high-performance neuroprosthesis for speech decoding and avatar control. Nature 620, 1037–1046 (2023).
Powell, M. P. et al. Epidural stimulation of the cervical spinal cord for post-stroke upper-limb paresis. Nat. Med. 29, 689–699 (2023).
Ortiz-Catalan, M., Mastinu, E., Sassu, P., Aszmann, O. & Brånemark, R. Self-contained neuromusculoskeletal arm prostheses. N. Engl. J. Med. 382, 1732–1738 (2020).
Graczyk, E. L., Resnik, L., Schiefer, M. A., Schmitt, M. S. & Tyler, D. J. Home use of a neural-connected sensory prosthesis provides the functional and psychosocial experience of having a hand again. Sci. Rep. 8, 9866 (2018).
Rubin, D. B. et al. Interim safety profile from the feasibility study of the braingate neural interface system. Neurology 100, e1177–e1192 (2023).
Greenspon, C. M. et al. Biomimetic multi-channel microstimulation of somatosensory cortex conveys high resolution force feedback for bionic hands. Preprint at bioRxiv https://doi.org/10.1101/2023.02.18.528972 (2023).
Valle, G. et al. Biomimetic computer-to-brain communication restoring naturalistic touch sensations via peripheral nerve stimulation. Preprint at https://doi.org/10.1101/2023.07.15.549130 (2023).
Valle, G. et al. Biomimetic intraneural sensory feedback enhances sensation naturalness, tactile sensitivity, and manual dexterity in a bidirectional prosthesis. Neuron 100, 37–45.e7 (2018).
Shelchkova, N. D. et al. Microstimulation of human somatosensory cortex evokes task-dependent, spatially patterned responses in motor cortex. Nat. Commun. 14, 7270 (2023).
George, J. A. et al. Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand. Sci. Robot. 4, eaax2352 (2019).
Katic, N. et al. Modeling foot sole cutaneous afferents: FootSim. iScience 26, 105874 (2023).
Saal, H. P., Delhaye, B. P., Rayhaun, B. C. & Bensmaia, S. J. Simulating tactile signals from the whole hand with millisecond precision. Proc. Natl Acad. Sci. USA 114, E5693–E5702 (2017).
Okorokova, E., He, Q. & Bensmaia, S. J. Biomimetic encoding model for restoring touch in bionic hands through a nerve interface. J. Neural Eng. 15, 066033 (2018).
Donati, E. & Indiveri, G. Neuromorphic bioelectronic medicine for nervous system interfaces: from neural computational primitives to medical applications. Prog. Biomed. Eng. 5, 013002 (2023).
Vitale, A., Donati, E., Germann, R. & Magno, M. Neuromorphic edge computing for biomedical applications: gesture classification using EMG signals. IEEE Sens. J. 22, 19490–19499 (2022).
Ceolini, E. et al. Hand-gesture recognition based on EMG and event-based camera sensor fusion: a benchmark in neuromorphic computing. Front. Neurosci. 14, 637 (2020).
Donati, E., Krause, R. & Indiveri, G. Neuromorphic pattern generation circuits for bioelectronic medicine. In 2021 10th International IEEE/EMBS Conference on Neural Engineering (NER) 1117–1120 (IEEE, 2021).
Krause, R. et al. Robust neuromorphic coupled oscillators for adaptive pacemakers. Sci. Rep. 11, 18073 (2021).
Bartolozzi, C. & Indiveri, G. Synaptic dynamics in analog VLSI. Neural Comput 19, 2581–2603 (2007).
Tian, Y. et al. Real-time, dynamic sensory feedback using neuromorphic tactile signals and transcutaneous electrical nerve stimulation. In 2022 IEEE Biomedical Circuits and Systems Conference (BioCAS) 399–403 (IEEE, 2022).
Spigler, G., Oddo, C. M. & Carrozza, M. C. Soft-neuromorphic artificial touch for applications in neuro-robotics. In 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) 1913–1918 (IEEE, 2012) https://doi.org/10.1109/BioRob.2012.6290849.
Lee, Y. et al. A low-power stretchable neuromorphic nerve with proprioceptive feedback. Nat. Biomed. Eng. 7, 511–519 (2023).
Chen, J. C. et al. Self-rectifying magnetoelectric metamaterials for remote neural stimulation and motor function restoration. Nat. Mater. 1–8 https://doi.org/10.1038/s41563-023-01680-4 (2023).
Kim, Y. et al. A bioinspired flexible organic artificial afferent nerve. Science 360, 998–1003 (2018).
Wang, W. et al. Neuromorphic sensorimotor loop embodied by monolithically integrated, low-voltage, soft e-skin. Science 380, 735–742 (2023).
Buccelli, S. et al. A neuromorphic prosthesis to restore communication in neuronal networks. iScience 19, 402–414 (2019).
Boi, F. et al. A bidirectional brain-machine interface featuring a neuromorphic hardware decoder. Front. Neurosci. 10, 563 (2016).
Broccard, F. D., Joshi, S., Wang, J. & Cauwenberghs, G. Neuromorphic Neural Interfaces. In Handbook of Neuroengineering (ed. Thakor, N. V.) 1–33 (Springer, 2020).
Pandarinath, C. & Bensmaia, S. J. The science and engineering behind sensitized brain-controlled bionic hands. Physiol. Rev. 102, 551–604 (2022).
Shen, K., Chen, O., Edmunds, J. L., Piech, D. K. & Maharbiz, M. M. Translational opportunities and challenges of invasive electrodes for neural interfaces. Nat. Biomed. Eng. 1–19 https://doi.org/10.1038/s41551-023-01021-5. (2023)
Raspopovic, S., Valle, G. & Petrini, F. M. Sensory feedback for limb prostheses in amputees. Nat. Mater. 1–15 https://doi.org/10.1038/s41563-021-00966-9 (2021).
Hochberg, L. R. et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375 (2012).
Collinger, J. L. et al. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381, 557–564 (2013).
Ajiboye, A. B. et al. Restoration of reaching and grasping in a person with tetraplegia through brain-controlled muscle stimulation: a proof-of-concept demonstration. Lancet 389, 1821–1830 (2017).
Vu, P. P. et al. A regenerative peripheral nerve interface allows real-time control of an artificial hand in upper limb amputees. Sci. Transl. Med. 12, eaay2857 (2020).
Wendelken, S. et al. Restoration of motor control and proprioceptive and cutaneous sensation in humans with prior upper-limb amputation via multiple Utah Slanted Electrode Arrays (USEAs) implanted in residual peripheral arm nerves. J. Neuroeng. Rehabil. 14, 121 (2017).
Rossini, P. M. et al. Double nerve intraneural interface implant on a human amputee for robotic hand control. Clin. Neurophysiol. 121, 777–783 (2010).
Petrini, F. M. et al. Six-month assessment of a hand prosthesis with intraneural tactile feedback. Ann. Neurol. 85, 137–154 (2019).
Tan, D. W. et al. A neural interface provides long-term stable natural touch perception. Sci. Transl. Med. 6, 257ra138 (2014).
Clites, T. R. et al. Proprioception from a neurally controlled lower-extremity prosthesis. Sci. Transl. Med. 10, eaap8373 (2018).
Petrini, F. M. et al. Sensory feedback restoration in leg amputees improves walking speed, metabolic cost and phantom pain. Nat. Med. 25, 1356–1363 (2019).
Charkhkar, H. et al. High-density peripheral nerve cuffs restore natural sensation to individuals with lower-limb amputations. J. Neural Eng. 15, 056002 (2018).
Overstreet, C. K., Cheng, J. & Keefer, E. Fascicle specific targeting for selective peripheral nerve stimulation. J. Neural Eng. 16, 066040 (2019).
Chandrasekaran, S. et al. Sensory restoration by epidural stimulation of the lateral spinal cord in upper-limb amputees. eLife 9, e54349 (2020).
Nanivadekar, A. C. et al. Restoration of sensory feedback from the foot and reduction of phantom limb pain via closed-loop spinal cord stimulation. Nat. Biomed. Eng. https://doi.org/10.1038/s41551-023-01153-8 (2023).
Flesher, S. N. et al. Intracortical microstimulation of human somatosensory cortex. Sci. Transl. Med. 8, 361ra141–361ra141 (2016).
Salas, M. A. et al. Proprioceptive and cutaneous sensations in humans elicited by intracortical microstimulation. eLife Sci. 7, e32904 (2018).
Fifer, M. S. et al. Intracortical somatosensory stimulation to elicit fingertip sensations in an individual with spinal cord injury. Neurology 98, e679–e687 (2022).
Valle, G. Peripheral neurostimulation for encoding artificial somatosensations. Eur. J. Neurosci. 56, 5888–5901 (2022).
Joung, Y.-H. Development of implantable medical devices: from an engineering perspective. Int. Neurourol. J. 17, 98–106 (2013).
Cutrone, A. & Micera, S. Implantable neural interfaces and wearable tactile systems for bidirectional neuroprosthetics systems. Adv. Healthc. Mater. 8, e1801345 (2019).
Oliva, N., de la, Navarro, X. & Valle, Jdel Time course study of long-term biocompatibility and foreign body reaction to intraneural polyimide-based implants. J. Biomed. Mater. Res. Part A 106, 746–757 (2018).
Christensen, M. B. et al. The foreign body response to the Utah Slant Electrode Array in the cat sciatic nerve. Acta Biomater. 10, 4650–4660 (2014).
Chee, L. et al. Cognitive benefits of using non-invasive compared to implantable neural feedback. Sci. Rep. 12, 16696 (2022).
Valle, G. et al. Hand control with invasive feedback is not impaired by increased cognitive load. Front. Bioeng. Biotechnol. 8, 287 (2020).
Raspopovic, S., Petrini, F. M., Zelechowski, M. & Valle, G. Framework for the development of neuroprostheses: from basic understanding by sciatic and median nerves models to bionic legs and hands. Proc. IEEE 105, 34–49 (2017).
Formento, E., D’Anna, E., Gribi, S., Lacour, S. P. & Micera, S. A biomimetic electrical stimulation strategy to induce asynchronous stochastic neural activity. J. Neural Eng. 17, 046019 (2020).
Negi, S., Bhandari, R., Rieth, L. & Solzbacher, F. In vitro comparison of sputtered iridium oxide and platinum-coated neural implantable microelectrode arrays. Biomed. Mater. 5, 15007 (2010).
Valle, G. et al. Multifaceted understanding of human nerve implants to design optimized electrodes for bioelectronics. Biomaterials 291, 121874 (2022).
Hughes, C. L. et al. Neural stimulation and recording performance in human sensorimotor cortex over 1500 days. J. Neural Eng. 18, 045012 (2021).
Wang, Y., Yang, X., Zhang, X., Wang, Y. & Pei, W. Implantable intracortical microelectrodes: reviewing the present with a focus on the future. Microsyst. Nanoeng. 9, 1–17 (2023).
Johansson, R. S. & Flanagan, J. R. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat. Rev. Neurosci. 10, 345–359 (2009).
Delhaye, B. P., Long, K. H. & Bensmaia, S. J. Neural basis of touch and proprioception in primate cortex. In Comprehensive Physiology (ed. Pollock, D. M.) 1575–1602 (John Wiley & Sons, Inc., 2018).
Saal, H. P. & Bensmaia, S. J. Touch is a team effort: interplay of submodalities in cutaneous sensibility. Trends Neurosci. 37, 689–697 (2014).
Callier, T., Suresh, A. K. & Bensmaia, S. J. Neural coding of contact events in somatosensory cortex. Cereb. Cortex 29, 4613–4627 (2019).
Saal, H. P. & Bensmaia, S. J. Biomimetic approaches to bionic touch through a peripheral nerve interface. Neuropsychologia 79, 344–353 (2015).
Hughes, C. & Kozai, T. Dynamic amplitude modulation of microstimulation evokes biomimetic onset and offset transients and reduces depression of evoked calcium responses in sensory cortices. Brain Stimul. 16, 939–965 (2023).
Torebjörk, H. E. & Ochoa, J. L. Specific sensations evoked by activity in single identified sensory units in man. Acta Physiol. Scand. 110, 445–447 (1980).
Davis, T. S. et al. Restoring motor control and sensory feedback in people with upper extremity amputations using arrays of 96 microelectrodes implanted in the median and ulnar nerves. J. Neural Eng. 13, 036001 (2016).
Dhillon, G. S. & Horch, K. W. Direct neural sensory feedback and control of a prosthetic arm. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 468–472 (2005).
Graczyk, E. L. et al. The neural basis of perceived intensity in natural and artificial touch. Sci. Transl. Med. 8, 362ra142–362ra142 (2016).
Ortiz-Catalan, M., Wessberg, J., Mastinu, E., Naber, A. & Branemark, R. Patterned stimulation of peripheral nerves produces natural sensations with regards to location but not quality. IEEE Trans. Med. Robot. Bionics 1, 199–203 (2019).
Indiveri, G. Introducing ‘neuromorphic computing and engineering’. Neuromorph. Comput. Eng. 1, 010401 (2021).
Mead, C. Neuromorphic electronic systems. Proc. IEEE 78, 1629–1636 (1990).
Indiveri, G. & Sandamirskaya, Y. The importance of space and time for signal processing in neuromorphic agents: the challenge of developing low-power, autonomous agents that interact with the environment. IEEE Signal Process. Mag. 36, 16–28 (2019).
Indiveri, G. & Liu, S.-C. Memory and information processing in neuromorphic systems. Proc. IEEE 103, 1379–1397 (2015).
Donati, E., Payvand, M., Risi, N., Krause, R. & Indiveri, G. Discrimination of EMG signals using a neuromorphic implementation of a spiking neural network. IEEE Trans. Biomed. Circuits Syst. 13, 795–803 (2019).
Yamazaki, K., Vo-Ho, V.-K., Bulsara, D. & Le, N. Spiking neural networks and their applications: a review. Brain Sci. 12, 863 (2022).
Mainen, Z. F. & Sejnowski, T. J. Reliability of spike timing in neocortical neurons. Science 268, 1503–1506 (1995).
Kaltenbrunner, M. et al. An ultra-lightweight design for imperceptible plastic electronics. Nature 499, 458–463 (2013).
Karipoth, P., Christou, A., Pullanchiyodan, A. & Dahiya, R. Bioinspired Inchworm- and earthworm-like soft robots with intrinsic strain sensing. Adv. Intell. Syst. 4, 2100092 (2022).
Ozioko, O. et al. SensAct: the soft and squishy tactile sensor with integrated flexible actuator. Adv. Intell. Syst. 3, 1900145 (2021).
Somlor, S., Hartanto, R. S., Schmitz, A. & Sugano, S. A novel tri-axial capacitive-type skin sensor. Adv. Robot. 29, 1375–1391 (2015).
Ward-Cherrier, B. et al. The TacTip Family: soft optical tactile sensors with 3D-printed biomimetic morphologies. Soft Robot. 5, 216–227 (2018).
Dahiya, R. S., Mittendorfer, P., Valle, M., Cheng, G. & Lumelsky, V. J. Directions toward effective utilization of tactile skin: a review. IEEE Sens. J. 13, 4121–4138 (2013).
Yogeswaran, N., Hosseini, E. S. & Dahiya, R. Graphene based low voltage field effect transistor coupled with biodegradable piezoelectric material based dynamic pressure sensor. ACS Appl. Mater. Interfaces 12, 54035–54040 (2020).
Hughes, D. & Correll, N. Texture recognition and localization in amorphous robotic skin. Bioinspir. Biomim. 10, 055002 (2015).
Lee, W. W., Kukreja, S. L. & Thakor, N. V. A kilohertz kilotaxel tactile sensor array for investigating spatiotemporal features in neuromorphic touch. In 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS) 1–4 (IEEE, 2015).
Dong, X. et al. Toward a living soft microrobot through optogenetic locomotion control of Caenorhabditis elegans. Sci. Robot. 6, eabe3950 (2021).
Izhikevich, E. M. Simple model of spiking neurons. IEEE Trans. Neural Netw. 14, 1569–1572 (2003).
Rongala, U. B., Mazzoni, A. & Oddo, C. M. Neuromorphic artificial touch for categorization of naturalistic textures. IEEE Trans. Neural Netw. Learn. Syst. 28, 819–829 (2017).
Lee, W. W. et al. A neuro-inspired artificial peripheral nervous system for scalable electronic skins. Sci. Robot. 4, eaax2198 (2019).
Osborn, L. E. et al. Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain. Sci. Robot. 3, https://doi.org/10.1126/scirobotics.aat3818 (2018).
Oddo, C. M. et al. Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans. Elife 5, e09148 (2016).
Zhengkun, Y. & Yilei, Z. Recognizing tactile surface roughness with a biomimetic fingertip: a soft neuromorphic approach. Neurocomputing 244, 102–111 (2017).
Friedl, K. E., Voelker, A. R., Peer, A. & Eliasmith, C. Human-inspired neurorobotic system for classifying surface textures by touch. IEEE Robot. Autom. Lett. 1, 516–523 (2016).
Rasouli, M., Chen, Y., Basu, A., Kukreja, S. L. & Thakor, N. V. An extreme learning machine-based neuromorphic tactile sensing system for texture recognition. IEEE Trans. Biomed. Circuits Syst. 12, 313–325 (2018).
Birkoben, T., Winterfeld, H., Fichtner, S., Petraru, A. & Kohlstedt, H. A spiking and adapting tactile sensor for neuromorphic applications. Sci. Rep. 10, 17260 (2020).
Bartolozzi, C., Indiveri, G. & Donati, E. Embodied neuromorphic intelligence. Nat. Commun. 13, 1024 (2022).
Caviglia, S., Pinna, L., Valle, M. & Bartolozzi, C. An event-driven POSFET taxel for sustained and transient sensing. In 2016 IEEE International Symposium on Circuits and Systems (ISCAS) 349–352 (IEEE, 2016).
Khalil, A. A., Valle, M., Chible, H. & Bartolozzi, C. CMOS dynamic tactile sensor. In 2017 New Generation of CAS (NGCAS) 269–272 (IEEE, 2017).
Janotte, E., Bamford, S., Richter, O., Valle, M. & Bartolozzi, C. Neuromorphic capacitive tactile sensors inspired by slowly adaptive mechanoreceptors. In 2022 20th IEEE Interregional NEWCAS Conference (NEWCAS) 119–123 (IEEE, 2022).
Kim, S., Lee, Y., Kim, H.-D. & Choi, S.-J. A tactile sensor system with sensory neurons and a perceptual synaptic network based on semivolatile carbon nanotube transistors. NPG Asia Mater. 12, 1–8 (2020).
Müller-Cleve, S. F. et al. Braille letter reading: A benchmark for spatio-temporal pattern recognition on neuromorphic hardware. Front. Neurosci. 16, 951164 (2022).
Chicca, E., Stefanini, F., Bartolozzi, C. & Indiveri, G. Neuromorphic electronic circuits for building autonomous cognitive systems. Proc. IEEE 102, 1367–1388 (2014).
Abu-Hassan, K. et al. Optimal solid state neurons. Nat. Commun. 10, 5309 (2019).
Rubino, A., Livanelioglu, C., Qiao, N., Payvand, M. & Indiveri, G. Ultra-low-power FDSOI neural circuits for extreme-edge neuromorphic intelligence. IEEE Trans. Circuits Syst. I: Regul. Pap. 68, 45–56 (2021).
Adam, K., Scholefield, A. & Vetterli, M. Sampling and reconstruction of bandlimited signals with multi-channel time encoding. IEEE Trans. Signal Process. 68, 1105–1119 (2020).
Deiss, S. R., Douglas, R. J. & Whatley, A. M. A pulse-coded communications infrastructure for neuromorphic systems. In Pulsed Neural Networks (eds Maass, W. & Bishop, C. M.) 157–178 (The MIT Press, 1999).
He, Y. et al. An implantable neuromorphic sensing system featuring near-sensor computation and send-on-delta transmission for wireless neural sensing of peripheral nerves. IEEE J. Solid-State Circuits 57, 3058–3070 (2022).
Alam, M. M., Torgo, L. & Bifet, A. A survey on spatio-temporal data analytics systems. ACM Comput. Surv. 54, 219:1–219:38 (2022).
Quaglio, P., Rostami, V., Torre, E. & Grün, S. Methods for identification of spike patterns in massively parallel spike trains. Biol. Cybern. 112, 57–80 (2018).
Posch, C., Matolin, D. & Wohlgenannt, R. A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS. IEEE J. Solid-State Circuits 46, 259–275 (2011).
Cohen, G. et al. Spatial and temporal downsampling in event-based visual classification. IEEE Trans. Neural Netw. Learn. Syst. 29, 5030–5044 (2018).
Oubari, O., Exarchakis, G., Lenz, G., Benosman, R. & Ieng, S.-H. Efficient spatio-temporal feature clustering for large event-based datasets. Neuromorph. Comput. Eng. 2, 044004 (2022).
Barak, O. Recurrent neural networks as versatile tools of neuroscience research. Curr. Opin. Neurobiol. 46, 1–6 (2017).
Sava, R., Donati, E. & Indiveri, G. Feed-forward and recurrent inhibition for compressing and classifying high dynamic range biosignals in spiking neural network architectures. Preprint at https://doi.org/10.48550/arXiv.2309.16425 (2023).
Manneschi, L. et al. Exploiting multiple timescales in hierarchical echo state networks. Front. Appl. Math. Stat. 6, https://doi.org/10.3389/fams.2020.616658 (2021).
Osborn, L. et al. Targeted transcutaneous electrical nerve stimulation for phantom limb sensory feedback. In 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS) 1–4 (IEEE, 2017).
Osborn, L., Kaliki, R. R., Soares, A. B. & Thakor, N. V. Neuromimetic event-based detection for closed-loop tactile feedback control of upper limb prostheses. IEEE Trans. Haptics 9, 196–206 (2016).
Mazzoni, A. et al. Morphological neural computation restores discrimination of naturalistic textures in trans-radial amputees. Sci. Rep. 10, 1–14 (2020).
Hughes, C. L. et al. Perception of microstimulation frequency in human somatosensory cortex. eLife 10, e65128 (2021).
Nielsen, C., Qiao, N. & Indiveri, G. A compact ultra low-power pulse delay and extension circuit for neuromorphic processors. In 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS) 1–4 (IEEE, 2017).
Cimolato, A., Ciotti, F., Kljajić, J., Valle, G. & Raspopovic, S. Symbiotic electroneural and musculoskeletal framework to encode proprioception via neurostimulation: ProprioStim. iScience 26, 106248 (2023).
Johnson, K. O. The roles and functions of cutaneous mechanoreceptors. Curr. Opin. Neurobiol. 11, 455–461 (2001).
Bisoni, L., Carboni, C., Raffo, L., Carta, N. & Barbaro, M. An HV-CMOS integrated circuit for neural stimulation in prosthetic applications. IEEE Trans. Circuits Syst. II: Express Briefs 62, 184–188 (2015).
Palomeque-Mangut, D., Rodríguez-Vázquez, Á. & Delgado-Restituto, M. Experimental validation of a high-voltage compliant neural stimulator implemented in a standard 1.8 V/3.3 V CMOS process. In 2022 IEEE Biomedical Circuits and Systems Conference (BioCAS) 335–339 (IEEE, 2022).
Ronchi, S. et al. Single-cell electrical stimulation using CMOS-based high-density microelectrode arrays. Front. Neurosci. 13, 208 (2019).
Liu, X., Demosthenous, A. & Donaldson, N. An Integrated Implantable Stimulator That Is Fail-safe Without Off-chip Blocking-capacitors. IEEE Trans. Biomed. Circuits Syst. 2, 231–244 (2008).
Kifle, Y. & Wikner, J. J. A reconfigurable 13.56 MHz wireless powered CMOS integrated nerve stimulator. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS) 956–959 (IEEE, 2022).
Weber, D. J., Friesen, R. & Miller, L. E. Interfacing the somatosensory system to restore touch and proprioception: essential considerations. J. Mot. Behav. 44, 403–418 (2012).
Capogrosso, M. et al. A computational model for epidural electrical stimulation of spinal sensorimotor circuits. J. Neurosci. 33, 19326–19340 (2013).
Nanivadekar, A. C., Ayers, C. A., Gaunt, R. A., Weber, D. J. & Fisher, L. E. Selectivity of afferent microstimulation at the DRG using epineural and penetrating electrode arrays. J. Neural Eng. 17, 016011 (2019).
Harrison, C., Epton, S., Bojanic, S., Green, A. L. & FitzGerald, J. J. The efficacy and safety of dorsal root ganglion stimulation as a treatment for neuropathic pain: a literature review. Neuromodulation 21, 225–233 (2018).
Liem, L. et al. A multicenter, prospective trial to assess the safety and performance of the spinal modulation dorsal root ganglion neurostimulator system in the treatment of chronic pain. Neuromodulation 16, 471–482 (2013).
Swan, B. D., Gasperson, L. B., Krucoff, M. O., Grill, W. M. & Turner, D. A. Sensory percepts induced by microwire array and DBS microstimulation in human sensory thalamus. Brain Stimul. 11, 416–422 (2018).
Romo, R., Hernández, A., Zainos, A. & Salinas, E. Somatosensory discrimination based on cortical microstimulation. Nature 392, 387–390 (1998).
Risso, G. & Valle, G. Multisensory integration in bionics: relevance and perspectives. Curr. Phys. Med. Rehabil. Rep. https://doi.org/10.1007/s40141-022-00350-x (2022).
Christie, B. P., Graczyk, E. L., Charkhkar, H., Tyler, D. J. & Triolo, R. J. Visuotactile synchrony of stimulation-induced sensation and natural somatosensation. J. Neural Eng. 16, 036025 (2019).
Christensen, M. B., Wark, H. A. C. & Hutchinson, D. T. A histological analysis of human median and ulnar nerves following implantation of Utah slanted electrode arrays. Biomaterials 77, 235–242 (2016).
De la Oliva, N., Navarro, X. & del Valle, J. Dexamethasone reduces the foreign body reaction to intraneural electrode implants in the peripheral nerve of the rat. Anat. Rec. 301, 1722–1733 (2018).
Aiello, G., Valle, G. & Raspopovic, S. Recalibration of neuromodulation parameters in neural implants with adaptive Bayesian optimization. J. Neural Eng. 20, 026037 (2023).
Prochazka, A. Proprioceptive feedback and movement regulation. In Comprehensive Physiology 89–127 (American Cancer Society, 2011).
Pillow, J. W. Prediction and decoding of retinal ganglion cell responses with a probabilistic spiking model. J. Neurosci. 25, 11003–11013 (2005).
Abbott, L. F. & Regehr, W. G. Synaptic computation. Nature 431, 796–803 (2004).
Zheng, Y. & Hu, X. Improved muscle activation using proximal nerve stimulation with subthreshold current pulses at kilohertz-frequency. J. Neural Eng. 15, 046001 (2018).
Valle, G. et al. Comparison of linear frequency and amplitude modulation for intraneural sensory feedback in bidirectional hand prostheses. Sci. Rep. 8, 16666 (2018).
Maimon, B. E. et al. Transdermal optogenetic peripheral nerve stimulation. J. Neural Eng. 14, 034002 (2017).
Cole, E. R. et al. Irregular optogenetic stimulation waveforms can induce naturalistic patterns of hippocampal spectral activity. Preprint at https://doi.org/10.1101/2022.09.21.508935 (2022).
Fumero, M. J. et al. A state-of-the-art implementation of a binaural cochlear-implant sound coding strategy inspired by the medial olivocochlear reflex. Hear. Res. 409, 108320 (2021).
Wiboonsaksakul, K. P., Roberts, D. C., Santina, C. C. D. & Cullen, K. E. A prosthesis utilizing natural vestibular encoding strategies improves sensorimotor performance in monkeys. PLoS Biol. 20, e3001798 (2022).
Bucciarelli, V. et al. Multiparametric non-linear TENS modulation to integrate intuitive sensory feedback. J. Neural Eng. 20, 036026 (2023).
Ciancio, A. L. et al. Control of prosthetic hands via the peripheral nervous system. Front. Neurosci. 10, 116 (2016).
Hahne, J. M., Schweisfurth, M. A., Koppe, M. & Farina, D. Simultaneous control of multiple functions of bionic hand prostheses: performance and robustness in end users. Sci. Robot. 3, eaat3630 (2018).
Hijjawi, J. B. et al. Improved myoelectric prosthesis control accomplished using multiple nerve transfers. Plast. Reconstr. Surg. 118, 1573–1578 (2006).
Kuiken, T. A. et al. Targeted reinnervation for enhanced prosthetic arm function in a woman with a proximal amputation: a case study. Lancet 369, 371–380 (2007).
Pasquina, P. F. et al. First-in-man demonstration of a fully implanted myoelectric sensors system to control an advanced electromechanical prosthetic hand. J. Neurosci. Methods 244, 85–93 (2015).
Salminger, S. et al. Long-term implant of intramuscular sensors and nerve transfers for wireless control of robotic arms in above-elbow amputees. Sci. Robot. 4, eaaw6306 (2019).
Farina, D. et al. The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges. IEEE Trans. Neural Syst. Rehabil. Eng. 22, 797–809 (2014).
Zhou, P. et al. Decoding a new neural–machine interface for control of artificial limbs. J. Neurophysiol. 98, 2974–2982 (2007).
Farina, D. et al. Man/machine interface based on the discharge timings of spinal motor neurons after targeted muscle reinnervation. Nat. Biomed. Eng. 1, 0025 (2017).
Orlandi, M. et al. Motor-unit ordering of blindly-separated surface-EMG signals for gesture recognition. In Advances in System-Integrated Intelligence (eds Valle, M. et al.) 518–529 (Springer International Publishing, 2023).
Cracchiolo, M. et al. Decoding of grasping tasks from intraneural recordings in trans-radial amputee. J. Neural Eng. 17, 026034 (2020).
Gesslbauer, B. et al. Axonal components of nerves innervating the human arm: arm nerve axonal components. Ann. Neurol. 82, 396–408 (2017).
Roche, A. D. et al. Clinical perspectives in upper limb prostheses: an update. Curr. Surg. Rep. 7, 5 (2019).
Roche, A. D., Rehbaum, H., Farina, D. & Aszmann, O. C. Prosthetic myoelectric control strategies: a clinical perspective. Curr. Surg. Rep. 2, 44 (2014).
Tsinganos, P., Cornelis, B., Jan, C., Jansen, B. & Skodras, A. Deep learning in EMG-based gesture recognition. In PhyCS 2018 - Proceedings of the 5th International Conference on Physiological Computing Systems (eds Dominguez Morales, M. J., Belda-Lois, J.-M., Pope, A., Placido da Silva, H. & Wang, C.) 107–114 https://doi.org/10.5220/0006960201070114 (2018).
Du, Y., Jin, W., Wei, W., Hu, Y. & Geng, W. Surface EMG-based inter-session gesture recognition enhanced by deep domain adaptation. Sensors (Basel) 17, 458 (2017).
Tateno, S., Liu, H. & Ou, J. Development of sign language motion recognition system for hearing-impaired people using electromyography signal. Sensors 20, 5807 (2020).
Zanghieri, M. et al. Robust real-time embedded EMG recognition framework using temporal convolutional networks on a multicore IoT processor. IEEE Trans. Biomed. Circuits Syst. 14, 244–256 (2020).
Krasoulis, A. & Nazarpour, K. Myoelectric digit action decoding with multi-output, multi-class classification: an offline analysis. Sci. Rep. 10, 16872 (2020).
Castellini, C. & van der Smagt, P. Surface EMG in advanced hand prosthetics. Biol. Cybern. 100, 35–47 (2009).
Hahne, J. M., Wilke, M. A., Koppe, M., Farina, D. & Schilling, A. F. Longitudinal case study of regression-based hand prosthesis control in daily life. Front. Neurosci. 14, 600 (2020).
Dhanasingh, A. & Jolly, C. An overview of cochlear implant electrode array designs. Hear. Res. 356, 93–103 (2017).
Reynolds, S. M. & Gifford, R. H. Effect of signal processing strategy and stimulation type on speech and auditory perception in adult cochlear implant users. Int. J. Audiol. 58, 363–372 (2019).
Tabibi, S., Kegel, A., Lai, W. K. & Dillier, N. A bio-inspired coding (BIC) strategy for cochlear implants. Hearing Res. 388, 107885 (2020).
Chan, V., Liu, S.-C. & van Schaik, A. AER EAR: a matched silicon cochlea pair with address event representation interface. IEEE Trans. Circuits Syst. I: Regul. Pap. 54, 48–59 (2007).
Jiménez-Fernández, A. et al. A binaural neuromorphic auditory sensor for FPGA: a spike signal processing approach. IEEE Trans. Neural Netw. Learn. Syst. 28, 804–818 (2017).
Lenk, C. et al. Neuromorphic acoustic sensing using an adaptive microelectromechanical cochlea with integrated feedback. Nat. Electron. 6, 370–380 (2023).
Fernandez, E. Development of visual Neuroprostheses: trends and challenges. Bioelectron. Med. 4, 12 (2018).
Ayton, L. N. et al. An update on retinal prostheses. Clin. Neurophysiol. 131, 1383–1398 (2020).
Vurro, M., Crowell, A. M. & Pezaris, J. S. Simulation of thalamic prosthetic vision: reading accuracy, speed, and acuity in sighted humans. Front. Hum. Neurosci. 8, 816 (2014).
Sakaguchi, H. et al. Artificial vision by direct optic nerve electrode (AV-DONE) implantation in a blind patient with retinitis pigmentosa. J. Artif. Organs 12, 206–209 (2009).
Troyk, P. R. et al. Intracortical visual prosthesis research—approach and progress. In 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference 7376–7379 (IEEE, 2005).
Lorach, H. et al. Artificial retina: the multichannel processing of the mammalian retina achieved with a neuromorphic asynchronous light acquisition device. J. Neural Eng. 9, 066004 (2012).
Park, J. H., Tan, J. S. Y., Wu, H., Dong, Y. & Yoo, J. 1225-Channel neuromorphic retinal-prosthesis SoC with localized temperature-regulation. IEEE Trans. Biomed. Circuits Syst. 14, 1230–1240 (2020).
Jameson, A., Chester, E. G. & Yakovlev, A. Address event representation neuromorphics to epiretinal colour vision. Electron. Lett. 49, 173–174 (2013).
Rueckauer, B. & van Gerven, M. Experiencing prosthetic vision with event-based sensors. In Proc. International Conference on Neuromorphic Systems 2022 1–7 (Association for Computing Machinery, 2022).
Chang, Y.-C., Ghaffari, D. H., Chow, R. H. & Weiland, J. D. Stimulation strategies for selective activation of retinal ganglion cell soma and threshold reduction. J. Neural Eng. 16, 026017 (2019).
Spencer, M. J., Kameneva, T., Grayden, D. B., Meffin, H. & Burkitt, A. N. Global activity shaping strategies for a retinal implant. J. Neural Eng. 16, 026008 (2019).
de Ruyter van Steveninck, J., Güçlü, U., van Wezel, R. & van Gerven, M. End-to-end optimization of prosthetic vision. J. Vis. 22, 20 (2022).
Granley, J., Relic, L. & Beyeler, M. Hybrid neural autoencoders for stimulus encoding in visual and other sensory neuroprostheses. Adv. Neural Inf. Process Syst. 35, 22671–22685 (2022).
Relic, L., Zhang, B., Tuan, Y.-L. & Beyeler, M. Deep learning–based perceptual stimulus encoder for bionic vision. In Proc. Augmented Humans International Conference 2022 323-325 (Association for Computing Machinery, 2022).
Bensmaia, S. J., Tyler, D. J. & Micera, S. Restoration of sensory information via bionic hands. Nat. Biomed. Eng. 1–13 https://doi.org/10.1038/s41551-020-00630-8 (2020).
Navarro, X. et al. A critical review of interfaces with the peripheral nervous system for the control of neuroprostheses and hybrid bionic systems. J. Peripher. Nerv. Syst. 10, 229–258 (2005).
WISE, K. D., ANDERSON, D. J., HETKE, J. F., KIPKE, D. R. & NAJAFI, K. Wireless implantable microsystems: high-density electronic interfaces to the nervous system. Proc. IEEE 92, 76–97 (2004).
Polikov, V. S., Tresco, P. A. & Reichert, W. M. Response of brain tissue to chronically implanted neural electrodes. J. Neurosci. Methods 148, 1–18 (2005).
Hoffmann, K. P., Koch, K. P., Doerge, T. & Micera, S. New technologies in manufacturing of different implantable microelectrodes as an interface to the peripheral nervous system. In Proc. of the First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006. 414–419 (IEEE, 2006).
Liu, Y. et al. Soft and elastic hydrogel-based microelectronics for localized low-voltage neuromodulation. Nat. Biomed. Eng. 3, 58–68 (2019).
Čvančara, P. et al. Bringing sensation to prosthetic hands—chronic assessment of implanted thin-film electrodes in humans. npj Flex Electron 7, 51 (2023).
Bourrier, A. et al. Monolayer graphene coating of intracortical probes for long-lasting neural activity monitoring. Adv. Healthc. Mater. 8, e1801331 (2019).
Tan, D. W., Schiefer, M. A., Keith, M. W., Anderson, J. R. & Tyler, D. J. Stability and selectivity of a chronic, multi-contact cuff electrode for sensory stimulation in human amputees. J. Neural Eng. 12, 026002 (2015).
Zelechowski, M., Valle, G. & Raspopovic, S. A computational model to design neural interfaces for lower-limb sensory neuroprostheses. J. Neuroeng. Rehabil. 17, 24 (2020).
Čvančara, P. et al. Stability of flexible thin-film metallization stimulation electrodes: analysis of explants after first-in-human study and improvement of in vivo performance. J. Neural Eng. 17, 046006 (2020).
Kubiak, C. A., Kemp, S. W. P. & Cederna, P. S. Regenerative peripheral nerve interface for management of postamputation neuroma. JAMA Surg. 153, 681–682 (2018).
Irwin, Z. T. et al. Chronic recording of hand prosthesis control signals via a regenerative peripheral nerve interface in a rhesus macaque. J. Neural Eng. 13, 046007 (2016).
Ursu, D. C., Urbanchek, M. G., Nedic, A., Cederna, P. S. & Gillespie, R. B. In vivo characterization of regenerative peripheral nerve interface function. J. Neural Eng. 13, 026012 (2016).
Woo, S. L. et al. Regenerative peripheral nerve interfaces for the treatment of postamputation neuroma pain: a Pilot Study. Plast. Reconstr. Surg. Glob. Open 4, e1038 (2016).
Go, G.-T., Lee, Y., Seo, D.-G. & Lee, T.-W. Organic neuroelectronics: from neural interfaces to neuroprosthetics. Adv. Mater. 34, 2201864 (2022).
Pehle, C. et al. The BrainScaleS-2 accelerated neuromorphic system with hybrid plasticity. Front. Neurosci. 16, 795876 (2022).
Cramer, B. et al. Surrogate gradients for analog neuromorphic computing. Proc. Natl Acad. Sci. USA 119, e2109194119 (2022).
Benjamin, B. V. et al. Neurogrid: a mixed-analog-digital multichip system for large-scale neural simulations. Proc. IEEE 102, 699–716 (2014).
Moradi, S., Qiao, N., Stefanini, F. & Indiveri, G. A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (DYNAPs). IEEE Trans. Biomed. Circuits Syst. 12, 106–122 (2018).
Neckar, A. et al. Braindrop: a mixed-signal neuromorphic architecture with a dynamical systems-based programming model. Proc. IEEE 107, 144–164 (2019).
Merolla, P. A. et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345, 668–673 (2014).
Davies, M. et al. Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro 38, 82–99 (2018).
Yan, Y. et al. Comparing Loihi with a SpiNNaker 2 prototype on low-latency keyword spotting and adaptive robotic control. Neuromorph. Comput. Eng. 1, 014002 (2021).
Höppner, S. et al. Adaptive body bias aware implementation for ultra-low-voltage designs in 22FDX technology. IEEE Trans. Circuits Syst. II: Express Briefs 67, 2159–2163 (2020).
Walter, D. et al. A 0.55 V 6.3uW/MHz arm cortex-M4 MCU with adaptive reverse body bias and single rail SRAM. In 2020 IEEE Symposium in Low-Power and High-Speed Chips (COOL CHIPS) 1–3 (IEEE, 2020).
Overview—Akida examples documentation https://doc.brainchipinc.com/.
Wark, B., Lundstrom, B. N. & Fairhall, A. Sensory adaptation. Curr. Opin. Neurobiol. 17, 423–429 (2007).
Laing, D. G. & Doty, R. L. The Human Sense of Smell (Springer-Verlag, 1991).
Leopold, D. A., Rhodes, G., Müller, K.-M. & Jeffery, L. The dynamics of visual adaptation to faces. Proc. Biol. Sci. 272, 897–904 (2005).
Grill-Spector, K., Henson, R. & Martin, A. Repetition and the brain: neural models of stimulus-specific effects. Trends Cogn. Sci. 10, 14–23 (2006).
Brenner, N., Bialek, W. & de Ruyter van Steveninck, R. Adaptive rescaling maximizes information transmission. Neuron 26, 695–702 (2000).
Graczyk, E. L., Delhaye, B. P., Schiefer, M. A., Bensmaia, S. J. & Tyler, D. J. Sensory adaptation to electrical stimulation of the somatosensory nerves. J. Neural Eng. 15, 046002 (2018).
Hughes, C. L., Flesher, S. N. & Gaunt, R. A. Effects of stimulus pulse rate on somatosensory adaptation in the human cortex. Brain Stimul. 15, 987–995 (2022).
Burgess, P. R. & Perl, E. R. Cutaneous mechanoreceptors and nociceptors. In Somatosensory System (eds. Albe-Fessard, D. et al.) 29–78 (Springer, 1973).
Chung, S., Li, X. & Nelson, S. B. Short-term depression at thalamocortical synapses contributes to rapid adaptation of cortical sensory responses in vivo. Neuron 34, 437–446 (2002).
Fink, A. J. P. et al. Presynaptic inhibition of spinal sensory feedback ensures smooth movement. Nature 509, 43–48 (2014).
Castellucci, V. F. & Kandel, E. R. A quantal analysis of the synaptic depression underlying habituation of the gill-withdrawal reflex in Aplysia. Proc. Natl Acad. Sci. USA 71, 5004–5008 (1974).
Sdrulla, A. D. et al. Electrical stimulation of low-threshold afferent fibers induces a prolonged synaptic depression in lamina II dorsal horn neurons to high-threshold afferent inputs in mice. Pain 156, 1008–1017 (2015).
Kilgore, K. L. & Bhadra, N. Nerve conduction block utilising high-frequency alternating current. Med. Biol. Eng. Comput. 42, 394–406 (2004).
Gemes, G. et al. Failure of action potential propagation in sensory neurons: mechanisms and loss of afferent filtering in C-type units after painful nerve injury. J. Physiol. 591, 1111–1131 (2013).
Klingner, C. M., Hasler, C., Brodoehl, S. & Witte, O. W. Excitatory and inhibitory mechanisms underlying somatosensory habituation. Hum. Brain Mapp. 35, 152–160 (2012).
Heiss, J. E., Katz, Y., Ganmor, E. & Lampl, I. Shift in the balance between excitation and inhibition during sensory adaptation of S1 neurons. J. Neurosci. 28, 13320–13330 (2008).
Kljajic, J., Valle, G. & Raspopovic, S. Modeling sensory adaptation to peripheral nerve stimulation. In 2021 10th International IEEE/EMBS Conference on Neural Engineering (NER) 788–791 (IEEE, 2021).
Eles, J. R., Stieger, K. C. & Kozai, T. D. Y. The temporal pattern of intracortical microstimulation pulses elicits distinct temporal and spatial recruitment of cortical neuropil and neurons. J. Neural. Eng. 18, https://doi.org/10.1088/1741-2552/abc29c (2021).
Kumaravelu, K., Sombeck, J., Miller, L. E., Bensmaia, S. J. & Grill, W. M. Stoney vs. Histed: quantifying the spatial effects of intracortical microstimulation. Brain Stimul. 15, 141–151 (2022).
Vallbo, A. B. & Johansson, R. S. Properties of cutaneous mechanoreceptors in the human hand related to touch sensation. Hum. Neurobiol. 3, 3–14 (1984).
Mitchell, D. E., Della Santina, C. C. & Cullen, K. E. Plasticity within non-cerebellar pathways rapidly shapes motor performance in vivo. Nat. Commun. 7, 11238 (2016).
Mongillo, G., Barak, O. & Tsodyks, M. Synaptic theory of working memory. Science 319, 1543–1546 (2008).
Acknowledgements
The authors thank Prof. Giacomo Indiveri and Dr. Chiara Bartolozzi for insightful discussions. The authors wish to thank Adrian M. Whatley for insightful comments on the manuscript and Ali Alamari for proofreading the manuscript. The funder had no role in the manuscript preparation or submission. The authors have received funding from the Chicago Postdoctoral Fellowship Program (no. 7-50488).
Author information
Authors and Affiliations
Contributions
E.D. and G.V. co-wrote this perspective and made the figures. Both authors edited and proofread the manuscript and authorized the submission of the manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Communications thanks the anonymous reviewers for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Donati, E., Valle, G. Neuromorphic hardware for somatosensory neuroprostheses. Nat Commun 15, 556 (2024). https://doi.org/10.1038/s41467-024-44723-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467-024-44723-3