Abstract
Neuromorphic computing using analog non-volatile memory (NVM) devices has been the subject of various studies due to its potential ability to achieve extremely low power consumption less than that of traditional von Neumann architecture. However, using NVM devices, such as phase change memory (PCM) and resistive-RAM devices, presents various challenges, such as limitations in the number of conductance steps and device variability resulting from process variation and electro/thermo-dynamics. Limitations in the number of conductance steps and device variability could reduce the accuracy of neural network training. It is necessary to quantitatively analyze the effect of the number of conductance steps and synaptic device variability on the accuracy of neural network training and assess requirements for NVM devices to make NVM-based neuromorphic computing successful. We conducted the analysis using simulations focusing on a spiking neural network (SNN) based restricted Boltzmann machine (RBM) with PCM devices using the PCM-refresh method. The results of our quantitative simulation, which used the MNIST dataset, showed that having more than 500 conductance steps achieves comparable performance to that when there are more than 1000 conductance steps. We also found that less than 10% conductance update variation in the synaptic devices is required to achieve the comparable accuracy with the no variation case. These results can provide guidelines for designing and optimizing a synaptic device for realizing NVM-based neuromorphic computing.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Burr GW, Shelby RM, Sebastian A, Kim S, Kim S, Sidler S, Virwani K, Ishii M, Narayanan P, Fumarola A et al (2017) Neuromorphic computing using non-volatile memory. Adv Phys X 2(1):89
Kim S, Ishii M, Lewis S, Perri T, BrightSky M, Kim W, Jordan R, Burr G, Sosa N, Ray A et al (2015) NVM neuromorphic core with 64k-cell (256-by-256) phase change memory synaptic array with on-chip neuron circuits for continuous in-situ learning, In: 2015 IEEE international electron devices meeting (IEDM). (IEEE), pp 17.1.1–17.1.4
Merolla PA, Arthur JV, Alvarez-Icaza R, Cassidy AS, Sawada J, Akopyan F, Jackson BL, Imam N, Guo C, Nakamura Y et al (2014) A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197):668
Davies M, Srinivasa N, Lin TH, Chinya G, Cao Y, Choday SH, Dimou G, Joshi P, Imam N, Jain S et al (2018) Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro 38(1):82
Ito M, Ishii M, Okazaki A, Kim S, Okazawa J, Nomura A, Hosokawa K, Haensch W (2018) Lightweight refresh method for PCM-based neuromorphic circuits. In: 2018 IEEE 18th international conference on nanotechnology (IEEE-NANO). (IEEE), pp 1–4
Bichler O, Suri M, Querlioz D, Vuillaume D, DeSalvo B, Gamrat C (2012) Visual pattern extraction using energy-efficient “2-PCM Synapse” neuromorphic architecture. IEEE Trans Electron Devices 59(8):2206
Zhang W, Li T (2009) Characterizing and mitigating the impact of process variations on phase change based memory systems. In: 2009 42nd Annual IEEE/ACM international symposium on microarchitecture (MICRO). (IEEE), pp 2–13
Suri M, Bichler O, Querlioz D, Cueto O, Perniola L, Sousa V, Vuillaume D, Gamrat C, DeSalvo B (2011) Phase change memory as synapse for ultra-dense neuromorphic systems: application to complex visual pattern extraction. In: 2011 IEEE international electron devices meeting (IEDM). (IEEE), pp 4.4.1–4.4.4
Kuzum D, Jeyasingh RG, Lee B, Wong HSP (2012) Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett 12(5):2179
Burr GW, Shelby RM, Sidler S, Di Nolfo C, Jang J, Boybat I, Shenoy RS, Narayanan P, Virwani K, Giacometti EU et al (2015) Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses) using phase-change memory as the synaptic weight element. IEEE Trans Electron Devices 62(11):3498
Ielmini D (2011) Modeling the universal set/reset characteristics of bipolar RRAM by field-and temperature-driven filament growth. IEEE Trans Electron Devices 58(12):4309
Neftci E, Das S, Pedroni B, Kreutz-Delgado K, Cauwenberghs G (2014) Event-driven contrastive divergence for spiking neuromorphic systems. Front Neurosci 7:272
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278
Nomura A, Ito M, Okazaki A, Ishii M, Kim S, Okazawa J, Hosokawa K, Haensch W (2018) NVM weight variation impact on analog spiking neural network chip. In International conference on neural information processing. (Springer), pp 676–685
Gokmen T, Vlasov Y (2016) Acceleration of deep neural network training with resistive cross-point devices: design considerations. Front Neurosci 10:333
Eryilmaz SB, Kuzum D, Yu S, Wong HSP (2015) Device and system level design considerations for analog-non-volatile-memory based neuromorphic architectures. In: 2015 IEEE international electron devices meeting (IEDM). (IEEE), pp 4.1.1–1.1.4
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Nomura, A., Ito, M., Okazaki, A. et al. Analysis of Effect of Weight Variation on SNN Chip with PCM-Refresh Method. Neural Process Lett 53, 1741–1751 (2021). https://doi.org/10.1007/s11063-019-10139-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-019-10139-0