Abstract
Deep spiking belief network (DSBN) uses unsupervised layer-wise pre-training method to train the network weights, it is stacked with the spike neural machine (SNM) modules. However, the synaptic weights of SNMs are difficult to pre-training through simple and effective approach for spike-train driven networks. This paper proposes a new algorithm that uses unsupervised multi-spike learning rule to train SNMs, which can implement the complex spatio-temporal pattern learning of spike trains. The spike signals first propagate in the forward direction, and then are reconstructed in the reverse direction, and the synaptic weights are adjusted according to the reconstruction error. The algorithm is successfully applied to spike train patterns, the module parameters are analyzed, such as the neuron number and learning rate in the SNMs. In addition, the low reconstruction errors of DSBNs are shown by the experimental results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)
Ghosh-Dastidar, S., Adeli, H.: Spiking neural networks. Int. J. Neural Syst. 19(4), 295–308 (2009)
Wang, X., Lin, X., Dang, X.: Supervised learning in spiking neural networks: a review of algorithms and evaluations. Neural Netw. 125, 258–280 (2020)
Kasabov, N., Feigin, V., Hou, Z.-G., et al.: Evolving spiking neural networks for personalised modelling, classification and prediction of spatio-temporal patterns with a case study on stroke. Neurocomputing 134, 269–279 (2014)
Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)
Lee, H., Grosse, R., Ranganath, R., et al.: Unsupervised learning of hierarchical representations with convolutional deep belief networks. Commun. ACM 54(10), 69–95 (2011)
Mleczko, W.K., Kapuściński, T., Nowicki, R.K.: Rough deep belief network - application to incomplete handwritten digits pattern classification. In: Dregvaite, G., Damasevicius, R. (eds.) ICIST 2015. CCIS, vol. 538, pp. 400–411. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24770-0_35
Kang, S., Qian, X., Meng, H.: Multi-distribution deep belief network for speech synthesis. In: IEEE International Conference on Acoustics, Speech and Signal processing, pp. 8012–8016. IEEE (2013)
Kuremoto, T., Kimura, S., Kobayashi, K., et al.: Time series forecasting using a deep belief network with restricted Boltzmann machines. Neurocomputing 137, 47–56 (2014)
Jo, T., Hou, J., Eickholt, J., et al.: Improving protein fold recognition by deep learning networks. Sci. Rep. 5, 17573 (2015)
O’Connor, P., Neil, D., Liu, S.-C., et al.: Real-time classification and sensor fusion with a spiking deep belief network. Front. Neurosci. 7, 1–13 (2013)
Tavanaei, A., Maida, A.S.: Studying the interaction of a hidden Markov model with a Bayesian spiking neural network. In: IEEE 25th International Workshop on Machine Learning for Signal Processing, pp. 1–6. IEEE (2015)
Neftci, E., Das, S., Pedroni, B., et al.: Event driven contrastive divergence for spiking neuromorphic systems. Front. Neurosci. 8, 1–14 (2014)
Fatahi, M., Shahsavari, M., Ahmadi, M., et al.: Rate-coded DBN: an online strategy for spike-based deep belief networks. Biol. Inspired Cogn. Arch. S2212683X18300173 (2018)
António, R.C., Park, I., Príncipe, J.C.: Inner products for representation and learning in the spike train domain. Stat. Signal Process. Neurosci. Neurotechnol., 265–309 (2010)
Paiva, A.R., Park, I., Príncipe, J.C.: A reproducing kernel Hilbert space framework for spike train signal processing. Neural Comput. 21(2), 424–449 (2009)
Lin, X., Wang, X., Hao, Z.: Supervised learning in multilayer spiking neural networks with inner products of spike trains. Neurocomputing 237, 59–70 (2017)
Mohemmed, A., Schliebs, S., Matsuda, S., et al.: SPAN: spike pattern association neuron for learning spatio-temporal spike patterns. Int. J. Neural Syst. 22, 1250012 (2012)
Gerstner, W., Kistler, W.M.: Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press, New York (2002)
Yu, Q., Tang, H., Tan, K.C., et al.: Precise-spike-driven synaptic plasticity: learning hetero-association of spatiotemporal spike patterns. PLoS ONE 8(11), e78318 (2013)
Tavanaei, A., Ghodrati, M., Kheradpisheh, S.R., et al.: Deep learning in spiking neural networks. Neural Netw. 111, 47–63 (2019)
Lin, X., Shi, G.: A supervised multi-spike learning algorithm for recurrent spiking neural networks. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11139, pp. 222–234. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01418-6_22
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lin, X., Du, P. (2020). Spike-Train Level Unsupervised Learning Algorithm for Deep Spiking Belief Networks. In: Farkaš, I., Masulli, P., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2020. ICANN 2020. Lecture Notes in Computer Science(), vol 12397. Springer, Cham. https://doi.org/10.1007/978-3-030-61616-8_51
Download citation
DOI: https://doi.org/10.1007/978-3-030-61616-8_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61615-1
Online ISBN: 978-3-030-61616-8
eBook Packages: Computer ScienceComputer Science (R0)