research-article

Public Access

Reliability Analysis of Memristive Reservoir Computing Architecture

Authors:

S. N. B. Tushar,

Garrett S. RoseAuthors Info & Claims

GLSVLSI '23: Proceedings of the Great Lakes Symposium on VLSI 2023

Pages 131 - 136

https://doi.org/10.1145/3583781.3590210

Published: 05 June 2023 Publication History

Abstract

Neuromorphic computing systems have emerged as powerful computation tools in the field of object recognition and control systems. However, training these systems, which are usually characterized by recurrent connectivity, requires abundant computational resources: memory, computation, data, and time. Reservoir computing (RC) framework reduces this high computational training cost by focusing the training effort on only a small subset of connections thus allowing these systems to be amenable to hardware implementation. Using memristors to construct these reservoir computers reduce the area/power consumption even further. However, the inherent variability of memristors poses specific challenges. Here, we conduct an in-depth reliability analysis of challenges posed by HfO2 memristors, including cycle-to-cycle variability, read/write noise, and conductance drift in the context of RC hardware. We also explore plasticity mechanisms such as Spike-Timing Dependent Plasticity (STDP) within the scope of the spiking recurrent neural networks (SRNN) reservoir and their impact on memristor conductance drift (MCD). We present a chaotic time series prediction task applied to a Python model of the constrained hardware design achieving very low Normalized Root Mean Square Error (NRMSE) of 2 × 10-3. The analog neuron and memristive synapse circuits employed for constructing the SRNN are simulated in Cadence Spectre and the energy consumption for the Mackey-Glass (MG) time-series prediction task was found to be approximately 90 nJ.

References

[1]

Hongyu An et al. 2020. Robust deep reservoir computing through reliable memristor with improved heat dissipation capability. IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems 40, 3 (2020), 574--583.

[2]

Guillaume Bellec et al. 2020. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature communications 11, 1 (2020), 3625.

[3]

Maxence Bouvier et al. 2019. Spiking neural networks hardware implementations and challenges: A survey. ACM Journal on Emerging Technologies in Computing Systems (JETC) 15, 2 (2019), 1--35.

Digital Library

[4]

Jens Bürger and Christof Teuscher. 2013. Variation-tolerant computing with memristive reservoirs. In International symposium on nanoscale architectures (NANOARCH 2013). IEEE/ACM, 1--6.

[5]

L. Chua. 1971. Memristor-The missing circuit element. IEEE Transactions on Circuit Theory (1971).

[6]

Leon Ong Chua. 2015. Everything You Wish to Know About Memristors but Are Afraid to Ask. In Handbook of Memristor Networks(2015).

[7]

Chao Du et al. 2017. Reservoir computing using dynamic memristors for temporal information processing. Nature communications 8, 1 (2017), 2204.

[8]

Manuel E López. 2020. Reliability-aware circuit design to mitigate impact of device defects and variability in emerging memristor-based applications. (2020).

[9]

Adam Z Foshie et al. 2022. Benchmark Comparisons of Spike-based Reconfigurable Neuroprocessor Architectures for Control Applications. In Proceedings of the Great Lakes Symposium on VLSI 2022. 383--386.

[10]

Daniel J Gauthier et al. 2021. Next generation reservoir computing. Nature communications 12, 1 (2021), 5564.

[11]

Geonhui Han et al. 2021. Alternative negative weight for simpler hardware implementation of synapse device based neuromorphic system. Scientific reports (2021).

[12]

Alex Henderson et al. 2022. Memristor Based Circuit Design for Liquid State Machine Verified with Temporal Classification. In International Joint Conference on Neural Networks (IJCNN 2022). IEEE, 1--9.

[13]

Stephen Alexander Henderson Jr. 2021. A Memristor-Based Liquid State Machine for Auditory Signal Recognition. Ph.D. Dissertation. University of Dayton.

[14]

Md Razuan Hossain et al. 2022. Memristor based Reservoir Network for Chaotic Time Series Prediction. In International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME 2022). IEEE, 1--6.

[15]

Herbert Jaeger. 2001. The "echo state" approach to analysing and training recurrent neural networks-with an erratum note. German National Research Center for Information Technology GMD Technical Report 148, 34 (2001), 13.

[16]

Herbert Jaeger, Wolfgang Maass, and Jose Principe. 2007. Special issue on echo state networks and liquid state machines. (2007).

[17]

Dhireesha Kudithipudi et al. 2016. Design and analysis of a neuromemristive reservoir computing architecture for biosignal processing. Frontiers in neuroscience 9 (2016), 502.

[18]

Boxun Li et al. 2014. ICE: Inline calibration for memristor crossbar-based computing engine. In Design, Automation & Test in Europe Conference & Exhibition (DATE 2014). IEEE.

[19]

Xiangpeng Liang et al. 2022. Rotating neurons for all-analog implementation of cyclic reservoir computing. Nature Communications 13 (03 2022).

[20]

Wolfgang Maass et al. 2002. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural computation 14, 11 (2002), 2531--2560.

[21]

Gianluca Milano et al. 2022. In materia reservoir computing with a fully memristive architecture based on self-organizing nanowire networks. Nature Materials 21, 2 (2022), 195--202.

[22]

Fabiha Nowshin. 2021. Spiking neural network with memristive based computingin- memory circuits and architecture. Ph.D. Dissertation. Virginia Tech.

[23]

Fabiha Nowshin et al. 2020. Recent Advances in Reservoir Computing With A Focus on Electronic Reservoirs. In International Green and Sustainable Computing Workshops (2020).

[24]

Manu Rathore et al. 2022. A Compact Model for the Variable Switching Dynamics of HfO 2 Memristors. In International Midwest Symposium on Circuits and Systems (MWSCAS 2022). IEEE.

[25]

Luís F Seoane. 2019. Evolutionary aspects of reservoir computing. Philosophical Transactions of the Royal Society B 374, 1774 (2019), 20180377.

[26]

Nicholas M Soures. 2017. Deep liquid state machines with neural plasticity and on-device learning. Rochester Institute of Technology.

[27]

Dmitri B Strukov et al. 2008. The missing memristor found. Nature (2008).

[28]

Gouhei Tanaka et al. 2019. Recent advances in physical reservoir computing: A review. Neural Networks 115 (2019), 100--123.

Digital Library

[29]

Pantelis R Vlachas et al. 2020. Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics. Neural Networks 126 (2020), 191--217.

[30]

Ryan Weiss. 2022. Hardware for Memristive Neuromorphic Systems with Reliable Programming and Online Learning. Ph.D. Dissertation.

[31]

Ryan Weiss et al. 2022. STDP Based Online Learning for a Current-Controlled Memristive Synapse. In International Midwest Symposium on Circuits and Systems (MWSCAS 2022). IEEE.

[32]

T Patrick Xiao et al. 2020. Analog architectures for neural network acceleration based on non-volatile memory. Applied Physics Reviews 7, 3 (2020), 031301.

[33]

Tadashi Yamazaki and Shigeru Tanaka. 2007. The cerebellum as a liquid state machine. Neural Networks 20, 3 (2007), 290--297.

Digital Library

[34]

Yanan Zhong et al. 2021. Dynamic memristor-based reservoir computing for high-efficiency temporal signal processing. Nature communications 12, 1 (2021).

Cited By

Rathore MRose G(2024)SpiCS-Net: Circuit Switched Network on Chip for Area-Efficient Spiking Recurrent Neural Networks2024 37th International Conference on VLSI Design and 2024 23rd International Conference on Embedded Systems (VLSID)10.1109/VLSID60093.2024.00040(204-209)Online publication date: 6-Jan-2024
https://doi.org/10.1109/VLSID60093.2024.00040
Rathore MRose G(2024)Leveraging Sparsity of SRNNs for Reconfigurable and Resource-Efficient Network-on-Chip2024 Neuro Inspired Computational Elements Conference (NICE)10.1109/NICE61972.2024.10548940(1-8)Online publication date: 23-Apr-2024
https://doi.org/10.1109/NICE61972.2024.10548940
Rathore MRose G(2024)Maximizing Efficiency of SNN-Based Reservoir Computing via NoC-Assisted Dimensionality Reduction2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI)10.1109/ISVLSI61997.2024.00128(671-674)Online publication date: 1-Jul-2024
https://doi.org/10.1109/ISVLSI61997.2024.00128
Show More Cited By

Index Terms

Reliability Analysis of Memristive Reservoir Computing Architecture

Recommendations

Delay learning and polychronization for reservoir computing

We propose a multi-timescale learning rule for spiking neuron networks, in the line of the recently emerging field of reservoir computing. The reservoir is a network model of spiking neurons, with random topology and driven by STDP (spike-time-dependent ...
Hardware design of LIF with Latency neuron model with memristive STDP synapses

In this paper, the hardware implementation of a neuromorphic system is presented. This system is composed of a Leaky Integrate-and-Fire with Latency (LIFL) neuron and a Spike-Timing Dependent Plasticity (STDP) synapse. LIFL neuron model allows to encode ...
2012 Special Issue: A forecast-based STDP rule suitable for neuromorphic implementation

Artificial neural networks increasingly involve spiking dynamics to permit greater computational efficiency. This becomes especially attractive for on-chip implementation using dedicated neuromorphic hardware. However, both spiking neural networks and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GLSVLSI '23: Proceedings of the Great Lakes Symposium on VLSI 2023

June 2023

731 pages

ISBN:9798400701252

DOI:10.1145/3583781

General Chairs:
Himanshu Thapliyal
University of Tennessee, Knoxville, USA
,
Ronald DeMara
University of Central Florida, USA
,
Program Chairs:
Inna Partin-Vaisband
University of Illinois Chicago, USA
,
Srinivas Katkoori
University of South Florida, USA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGDA: ACM Special Interest Group on Design Automation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 June 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Air Force Research Laboratory

Conference

GLSVLSI '23

Sponsor:

SIGDA

GLSVLSI '23: Great Lakes Symposium on VLSI 2023

June 5 - 7, 2023

TN, Knoxville, USA

Acceptance Rates

Overall Acceptance Rate 312 of 1,156 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
182
Total Downloads

Downloads (Last 12 months)92
Downloads (Last 6 weeks)12

Reflects downloads up to 28 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rathore MRose G(2024)SpiCS-Net: Circuit Switched Network on Chip for Area-Efficient Spiking Recurrent Neural Networks2024 37th International Conference on VLSI Design and 2024 23rd International Conference on Embedded Systems (VLSID)10.1109/VLSID60093.2024.00040(204-209)Online publication date: 6-Jan-2024
https://doi.org/10.1109/VLSID60093.2024.00040
Rathore MRose G(2024)Leveraging Sparsity of SRNNs for Reconfigurable and Resource-Efficient Network-on-Chip2024 Neuro Inspired Computational Elements Conference (NICE)10.1109/NICE61972.2024.10548940(1-8)Online publication date: 23-Apr-2024
https://doi.org/10.1109/NICE61972.2024.10548940
Rathore MRose G(2024)Maximizing Efficiency of SNN-Based Reservoir Computing via NoC-Assisted Dimensionality Reduction2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI)10.1109/ISVLSI61997.2024.00128(671-674)Online publication date: 1-Jul-2024
https://doi.org/10.1109/ISVLSI61997.2024.00128
Das HPatel KAmeli SChakraborty NSchuman CRose G(2024)Hardware-Application Co-Design to Evaluate the Performance of an STDP-based Reservoir Computer2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI)10.1109/ISVLSI61997.2024.00127(666-670)Online publication date: 1-Jul-2024
https://doi.org/10.1109/ISVLSI61997.2024.00127
Das HChakraborty NRathore MAlam SSchuman CRose G(2024)A Memristive Reconfigurable Neuromorphic Array for Neuro-Inspired Dynamic Architectures2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI)10.1109/ISVLSI61997.2024.00081(415-420)Online publication date: 1-Jul-2024
https://doi.org/10.1109/ISVLSI61997.2024.00081
Das HFahad ITushar SAlam SBuchanan GScott DRose GSwaminathan S(2024)In-Sensor Motion Recognition with Memristive System and Light Sensing Surfaces2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI)10.1109/ISVLSI61997.2024.00044(192-197)Online publication date: 1-Jul-2024
https://doi.org/10.1109/ISVLSI61997.2024.00044
Tushar SDas HRose G(2024) HfO 2 -Based Synaptic Spiking Neural Network Evaluation to Optimize Design and Testing Cost 2024 IEEE International Symposium on Circuits and Systems (ISCAS)10.1109/ISCAS58744.2024.10558518(1-5)Online publication date: 19-May-2024
https://doi.org/10.1109/ISCAS58744.2024.10558518
Rathore MRose G(2024)AnSpiCS-Net: Reconfigurable Network-on-Chip for Analog Spiking Recurrent Neural Networks2024 IEEE International Symposium on Circuits and Systems (ISCAS)10.1109/ISCAS58744.2024.10558368(1-5)Online publication date: 19-May-2024
https://doi.org/10.1109/ISCAS58744.2024.10558368
Das HSchuman CChakraborty NRose G(2024)Enhanced read resolution in reconfigurable memristive synapses for Spiking Neural NetworksScientific Reports10.1038/s41598-024-58947-214:1Online publication date: 17-Apr-2024
https://doi.org/10.1038/s41598-024-58947-2
Nair VReghuvaran CJohn DChoubey BJames A(2023)ESSM: Extended Synaptic Sampling Machine With Stochastic Echo State Neuro-Memristive CircuitsIEEE Journal on Emerging and Selected Topics in Circuits and Systems10.1109/JETCAS.2023.332887513:4(965-974)Online publication date: Dec-2023
https://doi.org/10.1109/JETCAS.2023.3328875

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents