tutorial

Mitigating Effects of Non-ideal Synaptic Device Characteristics for On-chip Learning

Authors:

Sarma Vrudhula,

Shimeng YuAuthors Info & Claims

ICCAD '15: Proceedings of the IEEE/ACM International Conference on Computer-Aided Design

Pages 194 - 199

Published: 02 November 2015 Publication History

Abstract

The cross-point array architecture with resistive synaptic devices has been proposed for on-chip implementation of weighted sum and weight update in the training process of learning algorithms. However, the non-ideal properties of the synaptic devices available today, such as the nonlinearity in weight update, limited ON/OFF range and device variations, can potentially hamper the learning accuracy. This paper focuses on the impact of these realistic properties on the learning accuracy and proposes the mitigation strategies. Unsupervised sparse coding is selected as a case study algorithm. With the calibration of the realistic synaptic behavior from the measured experimental data, our study shows that the recognition accuracy of MNIST handwriting digits degrades from ~97 % to ~65 %. To mitigate this accuracy loss, the proposed strategies include 1) the smart programming schemes for achieving linear weight update; 2) a dummy column to eliminate the off-state current; 3) the use of multiple cells for each weight element to alleviate the impact of device variations. With the improved synaptic behavior by these strategies, the accuracy increases back to ~95 %, enabling the reliable integration of realistic synaptic devices in the neuromorphic systems.

References

[1]

J. Hasler, et al., "Finding a roadmap to achieve large neuromorphic hardware systems," Frontiers in neuroscience, vol. 7, 2013.

[2]

J. Schemmel, et al., "A wafer-scale neuromorphic hardware system for large-scale neural modeling," IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1947--1950, 2010.

[3]

R. Serrano-Gotarredona, et al., "CAVIAR: A 45k neuron, 5M synapse, 12G connects/s AER hardware sensory--processing--learning--actuating system for high-speed visual object recognition and tracking," IEEE Transactions on Neural Networks, vol. 20, no. 9, pp. 1417--1438, 2009.

Digital Library

[4]

X. Jin, et al., "Modeling spiking neural networks on SpiNNaker," Computing in Science & Engineering, vol. 12, no. 5, pp. 91--97, 2010.

Digital Library

[5]

P. A. Merolla, et al., "A million spiking-neuron integrated circuit with a scalable communication network and interface," Science, vol. 345, no. 6197, pp. 668--673, 2014.

[6]

J. Partzsch, et al., "Analyzing the scaling of connectivity in neuromorphic hardware and in models of neural networks," IEEE Transactions on Neural Networks, vol. 22, no. 6, pp. 919--935, 2011.

Digital Library

[7]

D. Kuzum, et al., "Synaptic electronics: materials, devices and applications," Nanotechnology, vol. 24, no. 38, pp. 382001, 2013.

[8]

M. Hu, et al., "Memristor crossbar based hardware realization of BSB recall function," International Joint Conference on Neural Networks (IJCNN), pp. 1--7, 2012.

[9]

P. Gu, et al., "Technological exploration of rram crossbar array for matrix-vector multiplication," Asia and South Pacific Design Automation Conference (ASP-DAC), pp. 106--111, 2015.

[10]

D. Querlioz, et al., "Bioinspired networks with nanoscale memristive devices that combine the unsupervised and supervised learning approaches," International Symposium on Nanoscale Architectures (NANOARCH), pp. 203--210, 2012.

Digital Library

[11]

B. Liu, et al., "Reduction and IR-drop compensations techniques for reliable neuromorphic computing systems," International Conference on Computer-Aided Design (ICCAD), pp. 63--70, 2014.

Digital Library

[12]

P.-Y. Chen, et al., "Technology-design co-optimization of resistive cross-point array for accelerating learning algorithms on chip," Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 854--859, 2015.

Digital Library

[13]

I.-T. Wang, et al., "3D synaptic architecture with ultralow sub-10 fJ energy per spike for neuromorphic computation," International Electron Devices Meeting (IEDM), pp. 665--668, 2014.

[14]

S. Park, et al., "Neuromorphic speech systems using advanced ReRAM-based synapse," IEEE International Electron Device Meeting (IEDM), pp. 625--628, 2013.

[15]

S. H. Jo, et al., "Nanoscale memristor device as synapse in neuromorphic systems," Nano letters, vol. 10, no. 4, pp. 1297--1301, 2010.

[16]

H. Lee, et al., "Efficient sparse coding algorithms," Advances in neural information processing systems, pp. 801--808, 2006.

[17]

D. H. Hubel, et al., "Receptive fields, binocular interaction and functional architecture in the cat's visual cortex," The Journal of physiology, vol. 160, no. 1, pp. 106--154, 1962.

[18]

A. J. Bell, et al., "The "independent components" of natural scenes are edge filters," Vision research, vol. 37, no. 23, pp. 3327--3338, 1997.

[19]

Y. LeCun, et al., "The MNIST database of handwritten digits," 1998.

[20]

C.-C. Chang, et al., "LIBSVM: a library for support vector machines," ACM Transactions on Intelligent Systems and Technology (TIST), vol. 2, no. 3, pp. 27, 2011.

Digital Library

Cited By

Radhakrishnan AGopi AReghuvaran CJames A(2024)Variability-Aware Memristive Crossbars With ImageSplit Neural ArchitectureIEEE Transactions on Nanotechnology10.1109/TNANO.2024.337512523(274-280)Online publication date: 8-Mar-2024
https://dl.acm.org/doi/10.1109/TNANO.2024.3375125
Krishnan GSun JHazra JDu XLiehr MLi ZBeckmann KJoshi RCady NCao Y(2021)Robust RRAM-based In-Memory Computing in Light of Model Stability2021 IEEE International Reliability Physics Symposium (IRPS)10.1109/IRPS46558.2021.9405092(1-5)Online publication date: 21-Mar-2021
https://dl.acm.org/doi/10.1109/IRPS46558.2021.9405092
Jain SRaghunathan A(2019)CxDNNACM Transactions on Embedded Computing Systems10.1145/336203518:6(1-23)Online publication date: 15-Nov-2019
https://dl.acm.org/doi/10.1145/3362035
Show More Cited By

Index Terms

Mitigating Effects of Non-ideal Synaptic Device Characteristics for On-chip Learning

Recommendations

Technology-design co-optimization of resistive cross-point array for accelerating learning algorithms on chip
DATE '15: Proceedings of the 2015 Design, Automation & Test in Europe Conference & Exhibition

Technology-design co-optimization methodologies of the resistive cross-point array are proposed for implementing the machine learning algorithms on a chip. A novel read and write scheme is designed to accelerate the training process, which realizes ...
Impacts of device architecture and low current operation on resistive switching of HfOx nanoscale devices

A highly scaling feasibility of resistance memory with a via-hole structure including Ti/HfO"x is demonstrated in this work. An empirical model is used to predict the correlation between the forming voltage of 5nm-thick HfO"x devices with concave ...
Humidity-dependent synaptic characteristics in gelatin-based organic transistors
Abstract
Organic synaptic transistors based on gelatin gate dielectric exhibit humidity-dependent synaptic characteristics. The device showed excellent field-effect transistor behavior without hysteresis when measured in a vacuum and dry air, but clear ...
Graphical abstract

Display Omitted
Highlights
- Humidity-dependent synaptic characteristics were demonstrated in gelatin-based organic transistors.
- Experimental results revealed slow proton transportation through a sequence of hopping.
- Continuous and reversible conductance ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICCAD '15: Proceedings of the IEEE/ACM International Conference on Computer-Aided Design

November 2015

955 pages

ISBN:9781467383899

General Chair:
Diana Marculescu
Carnegie Mellon Univ., Dept. of Electrical and Computer Engineering, 5000 Forbes Ave., Pittsburgh, PA 15213, 412-268-1167
,
Program Chair:
Frank Liu
IBM Corp., IBM Austin Research Lab, 11501 Burnet Rd., MS 904-6G017, Austin, TX 78758, 512-286-7267

Sponsors

SIGDA: ACM Special Interest Group on Design Automation

Publisher

IEEE Press

Publication History

Published: 02 November 2015

Check for updates

Author Tags

Qualifiers

Tutorial
Research
Refereed limited

Conference

ICCAD '15

Sponsor:

SIGDA

ICCAD '15: IEEE/ACM INTERNATIONAL CONFERENCE ON COMPUTER-AIDED DESIGN

November 2 - 6, 2015

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 457 of 1,762 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
528
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)2

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Radhakrishnan AGopi AReghuvaran CJames A(2024)Variability-Aware Memristive Crossbars With ImageSplit Neural ArchitectureIEEE Transactions on Nanotechnology10.1109/TNANO.2024.337512523(274-280)Online publication date: 8-Mar-2024
https://dl.acm.org/doi/10.1109/TNANO.2024.3375125
Krishnan GSun JHazra JDu XLiehr MLi ZBeckmann KJoshi RCady NCao Y(2021)Robust RRAM-based In-Memory Computing in Light of Model Stability2021 IEEE International Reliability Physics Symposium (IRPS)10.1109/IRPS46558.2021.9405092(1-5)Online publication date: 21-Mar-2021
https://dl.acm.org/doi/10.1109/IRPS46558.2021.9405092
Jain SRaghunathan A(2019)CxDNNACM Transactions on Embedded Computing Systems10.1145/336203518:6(1-23)Online publication date: 15-Nov-2019
https://dl.acm.org/doi/10.1145/3362035
Wang CFeng DTong WLiu JLi ZChang JZhang YWu BXu JZhao WLi YRen R(2019)Cross-point Resistive MemoryACM Transactions on Design Automation of Electronic Systems10.1145/332506724:4(1-37)Online publication date: 20-Jun-2019
https://dl.acm.org/doi/10.1145/3325067
Chang CWu MLin JLi CParmar VLee HWei JSheu SSuri MChang THou T(2019)NV-BNNProceedings of the 56th Annual Design Automation Conference 201910.1145/3316781.3317872(1-6)Online publication date: 2-Jun-2019
https://dl.acm.org/doi/10.1145/3316781.3317872
Sun XPeng XChen PLiu RSeo JYu SShin Y(2018)Fully parallel RRAM synaptic array for implementing binary neural network with (+1, −1) weights and (+1, 0) neuronsProceedings of the 23rd Asia and South Pacific Design Automation Conference10.5555/3201607.3201741(574-579)Online publication date: 22-Jan-2018
https://dl.acm.org/doi/10.5555/3201607.3201741
Athreyas NSong WPerot BXia QMathew AGupta JGupta DYang J(2018)Memristor-CMOS Analog Coprocessor for Acceleration of High-Performance Computing ApplicationsACM Journal on Emerging Technologies in Computing Systems10.1145/326998514:3(1-30)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.1145/3269985
Narayanan PFumarola ASanches LHosokawa KLewis SShelby RBurr G(2017)Toward on-chip acceleration of the backpropagation algorithm using nonvolatile memoryIBM Journal of Research and Development10.1147/JRD.2017.271657961:4-5(11:1-11:11)Online publication date: 1-Jul-2017
https://dl.acm.org/doi/10.1147/JRD.2017.2716579
Xu ZSkorheim STu MBerisha VYu SSeo JBazhenov MCao Y(2017)Improving efficiency in sparse learning with the feedforward inhibitory motifNeurocomputing10.1016/j.neucom.2017.05.016267:C(141-151)Online publication date: 6-Dec-2017
https://dl.acm.org/doi/10.1016/j.neucom.2017.05.016

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents