Abstract
Deep neural networks have surpassed human performance in key visual challenges such as object recognition, but require a large amount of energy, computation, and memory. In contrast, spiking neural networks (SNNs) have the potential to improve both the efficiency and biological plausibility of object recognition systems. Here we present a SNN model that uses spike-latency coding and winner-take-all inhibition (WTA-I) to efficiently represent visual stimuli using multi-scale parallel processing. Mimicking neuronal response properties in early visual cortex, images were preprocessed with three different spatial frequency (SF) channels, before they were fed to a layer of spiking neurons whose synaptic weights were updated using spike-timing-dependent-plasticity. We investigate how the quality of the represented objects changes under different SF bands and WTA-I schemes. We demonstrate that a network of 200 spiking neurons tuned to three SFs can efficiently represent objects with as little as 15 spikes per neuron. Studying how core object recognition may be implemented using biologically plausible learning rules in SNNs may not only further our understanding of the brain, but also lead to novel and efficient artificial vision systems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ales JM, Appelbaum LG, Cottereau BR et al (2013) The time course of shape discrimination in the human brain. Neuroimage 67:77–88
Beyeler M, Dutt ND, Krichmar JL (2013) Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule. Neural Netw 48:109–24
Beyeler M, Dutt N, Krichmar JL (2016) 3D visual response properties of MSTd emerge from an efficient, sparse population code. J Neurosci 36(32):8399–8415
Beyeler M, Rounds E, Carlson K et al (2019) Neural correlates of sparse coding and dimensionality reduction. PLoS Comput Biol 15(6):e1006908
Bi GQ, Poo MM (1998) Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18(24):10,464-10,472
Bing Z, Baumann I, Jiang Z et al (2019) Supervised learning in snn via reward-modulated spike-timing-dependent plasticity for a target reaching vehicle. Front Neurorobot 13:18
Brzosko Z, Mierau SB, Paulsen O (2019) Neuromodulation of spike-timing-dependent plasticity: past, present, and future. Neuron 103(4):563–581
Campbell, Fergus W. The transmission of spatial information through the visual system. From Theoretical Physics to Biology. Karger Publishers, 1973. 374–384
Caporale N, Dan Y et al (2008) Spike timing-dependent plasticity: a hebbian learning rule. Annu Rev Neurosci 31(1):25–46
Chang L, Tsao DY (2017) The code for facial identity in the primate brain. Cell 169(6):1013–1028
Chauhan T, Masquelier T, Montlibert A et al (2018) Emergence of binocular disparity selectivity through Hebbian learning. J Neurosci 38(44):9563–9578
Chauhan T, Masquelier T, Cottereau BR (2021) Sub-optimality of the early visual system explained through biologically plausible plasticity. Front Neurosci 15:727448
Cichy RM, Pantazis D, Oliva A (2016) Similarity-based fusion of meg and fmri reveals spatio-temporal dynamics in human cortex during visual object recognition. Cereb Cortex 26(8):3563–3579
De Valois RL, Albrecht DG, Thorell LG (1982) Spatial frequency selectivity of cells in macaque visual cortex. Vis Res 22(5):545–559
De Valois RL, Albrecht DG, Thorell LG (1982) Spatial frequency selectivity of cells in macaque visual cortex. Vis Res 22(5):545–559
Delorme A, Thorpe SJ (2001) Face identification using one spike per neuron: resistance to image degradations. Neural Netw 14(6–7):795–803
Derrington A, Lennie P (1982) The influence of temporal frequency and adaptation level on receptive field organization of retinal ganglion cells in cat. J Physiol 333(1):343–366
Derrington A, Lennie P, Wright M (1979) The mechanism of peripherally evoked responses in retinal ganglion cells. J Physiol 289(1):299–310
DiCarlo J, Zoccolan D, Rust N (2012) How does the brain solve visual object recognition? Neuron 73(3):415–434
Diehl PU, Cook M (2015) Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front Comput Neurosci. https://doi.org/10.3389/fncom.2015.00099
Enroth-Cugell C, Robson JG (1966) The contrast sensitivity of retinal ganglion cells of the cat. J Physiol 187(3):517–552
Falez P, Tirilly P, Bilasco IM, et al (2019) Multi-layered spiking neural network with target timestamp threshold adaptation and stdp. In: 2019 international joint conference on neural networks (IJCNN). IEEE, pp 1–8
Feldman DE (2012) The spike-timing dependence of plasticity. Neuron 75(4):556–571
Field DJ (1987) Relations between the statistics of natural images and the response properties of cortical cells. Josa A 4(12):2379–2394
Fu Q, Dong H (2021) An ensemble unsupervised spiking neural network for objective recognition. Neurocomputing 419:47–58
Gerstner W, Kistler WM (2002) Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press, Cambridge
Ginsburg AP (1986) Spatial filtering and visual form perception. Handbook of Perception and Human Performance, Vol 2 Cognitive Processes and Performance
Goel A, Tung C, Lu YH, et al (2020) A survey of methods for low-power deep learning and computer vision. In: 2020 IEEE 6th world forum on internet of things (WF-IoT). IEEE, pp 1–6
Gütig R, Aharonov R, Rotter S et al (2003) Learning input correlations through nonlinear temporally asymmetric hebbian plasticity. J Neurosci 23(9):3697–3714. https://doi.org/10.1523/JNEUROSCI.23-09-03697.2003
Gütig R, Aharonov R, Rotter S et al (2003) Learning input correlations through nonlinear temporally asymmetric hebbian plasticity. J Neurosci 23(9):3697–3714
Hao Y, Huang X, Dong M et al (2020) A biologically plausible supervised learning method for spiking neural networks using the symmetric stdp rule. Neural Netw 121:387–395
He K, Zhang X, Ren S, et al (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034
Henriksson L, Nurminen L, Hyvärinen A et al (2008) Spatial frequency tuning in human retinotopic visual areas. J Vis 8(10):5–5
Hughes HC, Nozawa G, Kitterle F (1996) Global precedence, spatial frequency channels, and the statistics of natural images. J Cognit Neurosci 8(3):197–230
Jiang P, Ergu D, Liu F et al (2022) A review of yolo algorithm developments. Procedia Comput Sci 199:1066–1073
Kauffmann L, Ramanoël S, Peyrin C (2014) The neural bases of spatial frequency processing during scene perception. Front Integr Neurosci 8:37
Kheradpisheh SR, Ganjtabesh M, Thorpe SJ et al (2018) Stdp-based spiking deep convolutional neural networks for object recognition. Neural Netw 99:56–67
Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Tech. Rep. 0, University of Toronto, Toronto, Ontario
LeCun Y (1998) The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/
Liu D, Yue S (2016) Visual pattern recognition using unsupervised spike timing dependent plasticity learning. In: 2016 international joint conference on neural networks (IJCNN). IEEE, pp 285–292
Liu Q, Pan G, Ruan H et al (2020) Unsupervised aer object recognition based on multiscale spatio-temporal features and spiking neurons. IEEE Trans Neural Netw Learn Syst 31(12):5300–5311
Maass W (2000) On the computational power of winner-take-all. Neural Comput 12(11):2519–2535
Majaj NJ, Hong H, Solomon EA et al (2015) Simple learned weighted sums of inferior temporal neuronal firing rates accurately predict human core object recognition performance. J Neurosci 35(39):13,402-13,418
Masquelier T, Thorpe S (2007) Unsupervised learning of visual features through spike timing dependent plasticity. PLoS Comput Biol 3(2):e31
Mozafari M, Ganjtabesh M, Nowzari-Dalini A et al (2019) Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks. Pattern Recognit 94:87–95
Nassi JJ, Callaway EM (2009) Parallel processing strategies of the primate visual system. Nat Rev Neurosci 10(5):360–372
Olshausen BA, Field DJ (1997) Sparse coding with an overcomplete basis set: a strategy employed by V1? Vis Res 37(23):3311–3325. https://doi.org/10.1016/S0042-6989(97)00169-7
Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of 1994 IEEE workshop on applications of computer vision. IEEE, pp 138–142
Sanchez-Garcia M, Chauhan T, Cottereau BR, et al (2022) Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition. arXiv:2212.00081
Shapley R, Lennie P et al (1985) Spatial frequency analysis in the visual system. Annu Rev Neurosci 8(1):547–581
Solomon SG, White AJ, Martin PR (2002) Extraclassical receptive field properties of parvocellular, magnocellular, and koniocellular cells in the primate lateral geniculate nucleus. J Neurosci 22(1):338–349
Stivaktakis R, Tsagkatakis G, Tsakalides P (2019) Deep learning for multilabel land cover scene categorization using data augmentation. IEEE Geosci Remote Sens Lett 16(7):1031–1035
Stuijt J, Sifalakis M, Yousefzadeh A et al (2021) \(\mu \)brain: an event-driven and fully synthesizable architecture for spiking neural networks. Front Neurosci 15:538
Sun Y, Liang D, Wang X, et al (2015) Deepid3: face recognition with very deep neural networks. arXiv:1502.00873
Tolhurst DJ, Tadmor Y, Chao T (1992) Amplitude spectra of natural images. Ophthalmic Physiol Opt 12(2):229–232
Vigneron A, Martinet J (2020) A critical survey of stdp in spiking neural networks for pattern recognition. In: 2020 international joint conference on neural networks (IJCNN). IEEE, pp 1–9
Vinje WE, Gallant JL (2000) Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287(5456):1273–1276
Xiao H, Rasul K, Vollgraf R (2017) Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747
Yu Q, Tang H, Tan KC et al (2013) Rapid feedforward computation by temporal encoding and learning with spiking neurons. IEEE Trans Neural Netw Learn Syst 24(10):1539–1552
Zhou Q, Li X (2022) A bio-inspired hierarchical spiking neural network with reward-modulated stdp learning rule for aer object recognition. IEEE Sens J 22(16):16,323-16,338
Acknowledgements
This work was partially supported by a UCSB Academic Senate Faculty Research Grant to MB and by FLAG-ERA project JTC-2019 DOMINO to BRC. TC was partially supported by the grants DE-SC0022997 (US Department of Energy) and FRM:SPF20170938752 (Fondation pour la Recherche Médical, France), and a Picower Fellowship from The JPB Foundation.
Author information
Authors and Affiliations
Contributions
TC and BRC conceived and designed the original study, which was subsequently extended by MSG and MB. TC wrote all the code and MSG ran all the simulations. MSG and MB analyzed and interpreted the results. MSG drafted the manuscript. All authors reviewed and approved the final version of the manuscript.
Corresponding author
Additional information
Communicated by Benjamin Lindner.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Melani Sanchez-Garcia and Tushar Chauhan are co-first authors. Benoit R. Cottereau and Michael Beyeler are co-last authors.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Sanchez-Garcia, M., Chauhan, T., Cottereau, B.R. et al. Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition. Biol Cybern 117, 95–111 (2023). https://doi.org/10.1007/s00422-023-00956-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00422-023-00956-x