Abstract
The hardware circuit of neural network based on forgetting memristors not only has the characteristics of high computational efficiency and low power consumption, but also has the advantage that a memristor can store the weight of long-term memory and short-term memory. Neural networks based on forgetting memristors can process two different data sets; however, the number of data sets processed is determined by the conversion rate of short-term memory to long-term memory neural network. In this paper, a forgetting memristor model with controllable decay rate is proposed, the short-term memory and long-term memory of the long-term and short-term memory (LSTM) network based on forgetting memristor is proposed, and the conversion speed from short-term memory network to long-term memory network is controllable. In the process of transformation from short-term memory to long-term memory of LSTM network based on forgetting memristor, the decay rate of forgetting memristor can be controlled, and the duration of short-term memory of LSTM network can be set. A reset signal mechanism is proposed so that the state of short-term memory of LSTM network with high recognition rate can be controlled. Based on the proposed controllable decay rate and reset signal, the state of the short-term memory network with high recognition rate can be set, so the LSTM network with two states can realize the recognition of different number of images under different data sets. Finally, two kinds of data sets are tested on the LSTM network based on the forgetting memristor, and the recognition rate is good, which shows the effectiveness of the proposed algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
Enquiries about data availability should be directed to the authors.
References
Ali S, Bae J, Lee CH, Shin S, Kobayashi NP (2017) Ultra-low power non-volatile resistive crossbar memory based on pull up resistors. Org Electron 41:73–78
Burr GW, Shelby RM, Sebastian A, Kim S, Kim S, Sidler S, Virwani K, Ishii M, Narayanan P, Fumarola A et al (2017) Neuromorphic computing using non-volatile memory. Adv Phys: X 2(1):89–124
Cai F, Kumar S, Van Vaerenbergh T, Sheng X, Liu R, Li C, Liu Z, Foltin M, Yu S, Xia Q et al (2020) Power-efficient combinatorial optimization using intrinsic noise in memristor Hopfield neural networks. Nat Electron 3(7):409–418
Chen L, Li C, Huang T, Chen Y, Wen S, Qi J (2013) A synapse memristor model with forgetting effect. Phys Lett A 377(45–48):3260–3265
Chen L, Gong C, Li C, Huang J (2021a) Low power convolutional architectures: three operator switching systems based on forgetting memristor bridge. Sustain Cities Soc 69:102849
Chen L, Zhou W, Li C, Huang J (2021b) Forgetting memristors and memristor bridge synapses with long-and short-term memories. Neurocomputing 456:126–135
Chua L (1971) Memristor-the missing circuit element. IEEE Trans Circuit Theory 18(5):507–519
Dong T, Huang T (2019) Neural cryptography based on complex-valued neural network. IEEE Trans Neural Netw Learn Syst 31(11):4999–5004
Dong T, Xiang W, Huang T, Li H (2021) Pattern formation in a reaction–diffusion BAM neural network with time delay: \((k_1, k_2)\) mode Hopf-zero bifurcation case. IEEE Trans Neural Netw Learn Syst 33(12):7266–7276
Dong T, Gong X, Wang A, Li H, Huang T (2022) Data-driven tracking control for multi-agent systems with unknown dynamics via multithreading iterative q-learning. IEEE Trans Syst Man Cybern: Syst 53(4):2533–2542
Dou G, Zhao K, Guo M et al (2023)Memristor-based LSTM network for text classification[J]. Fractals 2023:2340040
Gao L, Chen P-Y, Yu S (2016) Demonstration of convolution kernel operation on resistive cross-point array. IEEE Electron Device Lett 37(7):870–873
Jo SH, Kim K-H, Lu W (2009) High-density crossbar arrays based on a Si memristive system. Nano Lett 9(2):870–874
Khaki S, Wang L, Archontoulis SV (2020) A CNN-RNN framework for crop yield prediction. Front Plant Sci 10:1750
Kim S, Choi B, Lim M, Yoon J, Lee J, Kim H-D, Choi S-J (2017) Pattern recognition using carbon nanotube synaptic transistors with an adjustable weight update protocol. ACS Nano 11(3):2814–2822
Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks[J]. Commun ACM 60(6):84–90
Kumar S, Strachan JP, Williams RS (2017) Chaotic dynamics in nanoscale NbO\(_2\) Mott memristors for analogue computing. Nature 548(7667):318–321
Lee C, Sarwar SS, Panda P et al (2020) Enabling spike-based backpropagation for training deep neural network architectures[J]. Frontiers Nurosci 2020:119
Li Y, Ang K-W (2021) Hardware implementation of neuromorphic computing using large-scale memristor crossbar arrays. Adv Intell Syst 3(1):2000137
Li Y, Zhong Y, Xu L, Zhang J, Xu X, Sun H, Miao X (2013) Ultrafast synaptic events in a chalcogenide memristor. Sci Rep 3(1):1–7
Li C, Wang Z, Rao M, Belkin D, Song W, Jiang H, Yan P, Li Y, Lin P, Hu M et al (2019) Long short-term memory networks in memristor crossbar arrays. Nat Mach Intell 1(1):49–57
Li S, Pam M-E, Li Y, Chen L, Chien Y-C, Fong X, Chi D, Ang K-W (2022) Wafer-scale 2D hafnium diselenide based memristor crossbar array for energy-efficient neural network hardware. Adv Mater 34(25):2103376
Lin H, Wang C, Hong Q, Sun Y (2020a) A multi-stable memristor and its application in a neural network. IEEE Trans Circuits Syst II: Express Briefs 67(12):3472–3476
Lin P, Li C, Wang Z, Li Y, Jiang H, Song W, Rao M, Zhuo Y, Upadhyay NK, Barnell M et al (2020b) Three-dimensional memristor circuits as complex neural networks. Nat Electron 3(4):225–232
Liu X, Zeng Z, Wunsch DC II (2020) Memristor-based LSTM network with in situ training and its applications. Neural Netw 131:300–311
Nguyen GN, Le Viet NH, Elhoseny M, Shankar K, Gupta B, Abd El-Latif AA (2021) Secure blockchain enabled cyber-physical systems in healthcare using deep belief network with ResNet model. J Parallel Distrib Comput 153:150–160
Roska T, Chua LO (1993) The CNN universal machine: an analogic array computer. IEEE Trans Circuits Syst II: Analog Digit Signal Process 40(3):163–173
Shamsi J, Amirsoleimani A, Mirzakuchaki S, Ahmadi M (2017) Modular neuron comprises of memristor-based synapse. Neural Comput Appl 28:1–11
Shang M, Wang X (2020) A memristor-based circuit design for generalization and differentiation on Pavlov associative memory. Neurocomputing 389:18–26
Sherstinsky A (2020) Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Phys D: Nonlinear Phenom 404:132306
Shi J, Zeng Z (2020) Design of in-situ learning bidirectional associative memory neural network circuit with memristor synapse. IEEE Trans Emerg Top Comput Intell 5(5):743–754
Strukov DB, Snider GS, Stewart DR, Williams RS (2008) The missing memristor found. Nature 453(7191):80–83
Sun K, Chen J, Yan X (2021) The future of memristors: materials engineering and neural networks. Adv Funct Mater 31(8):2006773
Sun J, Wang Y, Liu P et al (2022) Memristor-based neural network circuit with multimode generalization and differentiation on pavlov associative memory[J]. IEEE Trans Cyber 53(5):3351–3362
Taherkhani A, Belatreche A, Li Y, Cosma G, Maguire LP, McGinnity TM (2020) A review of learning in biologically plausible spiking neural networks. Neural Netw 122:253–272
Wang Z-R, Li Y, Su Y-T, Zhou Y-X, Cheng L, Chang T-C, Xue K-H, Sze SM, Miao X-S (2018) Efficient implementation of Boolean and full-adder functions with 1T1R RRAMs for beyond von Neumann in-memory computing. IEEE Trans Electron Devices 65(10):4659–4666
Wang T-Y, Meng J-L, Rao M-Y, He Z-Y, Chen L, Zhu H, Sun Q-Q, Ding S-J, Bao W-Z, Zhou P et al (2020) Three-dimensional nanoscale flexible memristor networks with ultralow power for information transmission and processing application. Nano Lett 20(6):4111–4120
Wang Z, Wang X, Zeng Z (2021) Memristive circuit design of brain-like emotional learning and generation. IEEE Trans Cybern 53(1):222–235
Wen S, Hu R, Yang Y, Huang T, Zeng Z, Song Y-D (2018a) Memristor-based echo state network with online least mean square. IEEE Trans Syst Man Cybern: Syst 49(9):1787–1796
Wen S, Xie X, Yan Z, Huang T, Zeng Z (2018b) General memristor with applications in multilayer neural networks. Neural Netw 103:142–149
Wen S, Wei H, Yang Y, Guo Z, Zeng Z, Huang T, Chen Y (2019) Memristive LSTM network for sentiment analysis. IEEE Trans Syst Man Cybern: Syst 51(3):1794–1804
Woo J, Moon K, Song J, Lee S, Kwak M, Park J, Hwang H (2016) Improved synaptic behavior under identical pulses using AlO\(_x\)/HfO\(_2\) bilayer RRAM array for neuromorphic systems. IEEE Electron Device Lett 37(8):994–997
Wu Z, Shen C, Van Den Hengel A (2019) Wider or deeper: revisiting the ResNet model for visual recognition. Pattern Recognit 90:119–133
Xie L, Du Nguyen HA, Taouil M, Hamdioui S, Bertels K (2017) A mapping methodology of Boolean logic circuits on memristor crossbar. IEEE Trans Comput Aided Des Integr Circuits Syst 37(2):311–323
Yakopcic C, Alom MZ, Taha TM (2016) Memristor crossbar deep network implementation based on a convolutional neural network[C]. In: International joint conference on neural networks (IJCNN). IEEE, pp 963–970
Yang Z, Wang X (2021) Memristor-based BAM circuit implementation for image associative memory and filling-in. Neural Comput Appl 33:7929–7942
Yao P, Wu H, Gao B, Tang J, Zhang Q, Zhang W, Yang JJ, Qian H (2020) Fully hardware-implemented memristor convolutional neural network. Nature 577(7792):641–646
Zeng X, Wen S, Zeng Z, Huang T (2018) Design of memristor-based image convolution calculation in convolutional neural network. Neural Comput Appl 30(2):503–508
Zhang X, Huang A, Hu Q, Xiao Z, Chu PK (2018) Neuromorphic computing with memristor crossbar. Phys Status Solidi (A) 215(13):1700875
Zhou T, Chen M, Wang Y, He J, Yang C (2020) Information entropy-based intention prediction of aerial targets under uncertain and incomplete information. Entropy 22(3):279
Acknowledgements
This paper is supported by National Key Research and Development Project (2018AAA0100101), Chongqing Key Laboratory of Mobile Communications Technology (cqupt-mct-202002), Research Programs of Chongqing Municipal Education Commission (Grant no. KJQN202300207), Chongqing higher education teaching reform (Grant no. 233094).
Funding
The authors have not disclosed any funding.
Author information
Authors and Affiliations
Contributions
YL and LC conceived the idea of the study; YL, LC and CL analyzed the data; YL, WZ and KL interpreted the results; YL wrote the paper; all the authors discussed the results and revised the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declared no potential conflicts of interest with respect to the research, authorship, and publication of this article.
Ethics and morality
This study does not involve ethics and morality.
Informed consent
Written informed consent was obtained from all the participants for the publication of this study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liu, Y., Chen, L., Li, C. et al. Long-term and short-term memory networks based on forgetting memristors. Soft Comput 27, 18403–18418 (2023). https://doi.org/10.1007/s00500-023-09110-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-023-09110-y