Abstract
Hebbian cell assemblies can be formalised as sets of tightly connected cells in auto- and hetero-associative memories. Direct evidence for such “cliques” has recently been obtained in multiple-unit recordings from rat hippocampal neurons. These experiments suggest a hierarchical organisation where cliques are embedded in each other such that larger cliques represent less specific stimulus conditions. We here suggest an interpretation stating that the firing patterns may not just reflect nested categories but a lattice of concepts about stimulus–response mappings in the sense of formal concept analysis, an applied branch of set theory. We present an implementation of formal concept lattices in bidirectional associative memories that in contrast to previous work satisfies Dale’s principle and uses balanced excitation and inhibition. Inhibitory cells have fixed, non-plastic synapses even if the model learns new concepts. As an extreme case a single global inhibitory cell is enough that controls the total level of activation. The excitatory cells can further learn incrementally using a Hebbian coincidence learning rule. Implications of the model for retrieval in auto-associative memories are further outlined. Overall the model is well suited for representing hierarchical compositional relationships between entities in the form of correlated patterns in technical cognitive systems and potentially the brain.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
We mainly follow Bělohlávek’s notation and definitions in [5] in the present work.
These entities may be features or objects or just anything, we do not make such distinctions in the auto-associative setup.
References
Abbott LF, Nelson SB. Synaptic plasticity: taming the beast. Nat Neurosci. 2000;3:1178–83.
Amit DJ. Modeling brain function. Cambridge: Cambridge University Press; 1988.
Amit DJ. The Hebbian paradigm reintegrated: local reverberations as internal representations. Behav Brain Sci. 1995;18:617–57.
Bi G-q, Poo M-m. Synaptic modification by correlated activity: Hebb’s postulate revisited. Annu Rev Neurosci. 2001;24:139–66.
Bělohlávek R. Representation of concept lattices by bidirectional associative memories. Neural Comput. 2000;12:2279–90.
Braitenberg V. Cell assemblies in the cerebral cortex. In: Heim R, Palm G, editors. Theoretical approaches to complex systems. Berlin: Springer; 1978. p. 171–88.
Braitenberg V, Schüz A. Anatomy of the cortex. Berlin: Springer-Verlag; 1991.
Dayan P. Images, frames, and connectionist hierarchies. Neural Comput. 2006;18:2293–319.
Eccles JC, Fatt P, Koketsu K. Cholinergic and inhibitory synapses in a pathway from motor-axon collaterals to motoneurones. J Physiol (Lond). 1954;126:52462.
Feldman J. Minimization of boolean complexity in human concept learning. Nature. 2000;407:630–3.
Freedman DJ, Riesenhuber M, Poggio T, Miller EK. A comparison of primate prefrontal and inferior temporal cortices during visual categorization. J Neurosci. 2003;23:5235–46.
Friston K. Hierarchical models in the brain. PLoS Comput Biol. 2008;4:e1000211.
Fukushima K. Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern. 1980;36:193–202.
Fukushima K. Restoring partly occluded patterns: a neural network model. Neural Netw. 2005;18:33–43.
Fukushima K, Myaki S. Neocognitron: a new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recognit 1982;15:455–69.
Fusi S, Abbott LF. Limits on the memory storage capacity of bounded synapses. Nat Neurosci. 2007;10:485–93.
Fusi S, Annunziato M, Badoni D, Salamon A, Amit DJ. Spike-driven synaptic plasticity: theory, simulation, VLSI implementation. Neural Comput. 2000;12:2227–58.
Fusi S, Drew PJ, Abbott LF. Cascade models of synaptically stored memories. Neuron. 2005;45:599–611.
Fuster JM. Memory in the cerebral cortex. An empirical approach to neural networks in the human and nonhuman primate. Cambridge, MA:MIT Press; 1994.
Ganter B, Wille R. Formal concept analysis: mathematical foundations. Berlin: Springer-Verlag; 1999.
Harris KD, Csicsvari J, Hirase H, Dragoi G, Buzsaki G. Organization of cell assemblies in the hippocampus. Nature. 2003;424:552–5.
Hebb D. The organization of behavior. New York: Wiley; 1949.
Hertz J, Krogh A, Palmer RG. Introduction to the theory of neural computation. Redwood city: Addison Wesley; 1991.
Hopfield JJ. Neural networks and physical systems with emergent collective computational abilities. PNAS. 1982;79:2554–8.
Indiveri G. Synaptic plasticity and spike-based computation in VLSI networks of integrate-and-fire neurons. Neural Inf Process—Lett Rev. 2007;11:135–46.
Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20:1254–1259
Kiani R, Esteky H, Mirpour K, Tanaka K. Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. J Neurophysiol. 2007;97(6):4296–309.
Kirk U. The neural basis of object-context relationships on aesthetic judgment. PLoS ONE. 2008;3:e3754.
Knoblauch A, Markert H, Palm GG. An associative cortical model of language understanding and action planning. In: Mira J, Alvarez JR, editors. Proceedings of IWINAC 2005, First International Work-Conference on the Interplay between natural and artificial computation, Las Palmas de Gran Canaria, Spain. vol. 3562, Lecture notes in computer science. Berlin, New York: Springer; 2005. p. 405–14.
Knoblauch A, Kupper R, Gewaltig M-O, Körner U, Körner E. A cell assembly based model for the cortical microcircuitry. Neurocomputing. 2007;70:1838–42.
Kosko B. Bidirectional associative memory. IEEE Trans Syst Man Cybern. 1988;18:49–60.
Lin L, Osan R, Shoham S, Jin W, Zuo W, Tsien JZ. Identification of network-level coding units for real-time representation of episodic experiences in the hippocampus. Proc Natl Acad Sci USA. 2005;102:6125–30.
Lin L, Osan R, Tsien JZ. Organizing principles of real-time memory encoding: neural clique assemblies and universal neural codes. Trends Neurosci. 2006;29:48–57.
Logothetis NK, Sheinberg DL. Visual object recognition. Annu Rev Neurosci. 1996;19:577–621.
Martinovic J, Gruber T, Müller MM. Coding of visual object features and feature conjunctions in the human brain. PLoS One. 2008;3:3781.
Miyashita Y, Chang HS. Neural correlate of pictorial short term memory. Nature. 1988;331:68–70.
Nosofsky RM, Gluck MA, Palmeri TJ, McKinley SC, Clautier P. Comparing models of rule-based classification learning: a replication and extension of Shepard, Hovland and Jenkins (1961). Mem Cognit. 1994;22:352–69.
Palm G. Neural assemblies: an alternative approach to artificial intelligence. Berlin: Springer-Verlag; 1982.
Palm G. Memory capacities of local rules for synaptic modification. Concepts Neurosci. 1991;2:97–128.
Palm G, Sommer F. Information capacity in recurrent McCulloch-Pitts networks with sparsely coded memory states. Network. 1992;3:177ff.
Pulvermüller F. Constituents of a neurological theory of language. Concepts Neurosci. 1992;3:157–200.
Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I. Invariant visual representation by single neurons in the human brain. Nature. 2005;435:1102–7.
Quiroga RQ, Kreimann G, Koch C, Fried I. Sparse but not ’grandmother-cell’ coding in the medial temporal lobe. Trends Cogn Sci. 2008(a);12:87–91.
Quiroga RQ, Mukamel R, Malach EA, Fried I. Human single-neuron responses at the threshold of conscious recognition. Proc Natl Acad Sci USA. 2008(b);105:3599–604.
Rajapakse R, Denham M. Fast access to concepts in concept lattices via bidirectional associative memories. Neural Comput. 2005;17:2291–300.
Rao RPN, Ballard DH. Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Comput. 1997;9:721–63.
Raz A, Buhle J. Typologies of attentional networks. Nat Rev Neurosci. 2006;7:367–79.
Riesenhuber M, Poggio T. Hierarchical models of object recognition in cortex. Nat Neurosci. 1999;2(11):1019–25.
Saalmann YB, Pigarev IN, Vidyasagar TR. Neural mechanisms of visual attention: how top-down feedback highlights relevant locations. Science. 2007;316:1612–5.
Schemmel J, Meier K, Mueller E. A new VLSI model of neural microcircuits including spike time dependent plasticity. In: Proceedings IJCNN. IEEE Press; 2007. p. 1711–6.
Shepard R, Hovland CL, Jenkins HM. Learning and memorization of classifications. Psychol Monogr Gen Appl. 1961;75:1–42.
Sloman SA, Rips LJ. Similarity and symbols in human thinking. Cambridge, MA: MIT Press; 1998.
Tanaka K. Neuronal mechanisms of object recognition. Science. 1993;262:685–8.
Tsunoda K, Yamane Y, Nishizaki M, Tanifuji M. Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns. Nat Neurosci. 2001;4:832–8.
Wennekers T. Operational cell assemblies as a paradigm for brain-inspired future computing architectures. Neural Inf Process—Lett Rev. 2006;10:135–45.
Wennekers T, Palm G. Cell assemblies, associative memory, and temporal structure in brain signals. In: Miller R, editors. Time and the brain. vol. 3, CABR—conceptual advances in brain research. Harwood Academic Publishers; 2000. p. 251–73.
Wennekers T, Palm G. Modelling generic cognitive functions with operational Hebbian cell assemblies. In: Weiss ML, editors. Neural network research horizons. Nova Science Publishers; 2007. p. 225–94.
Wennekers T, Garagnani M, Pulvermüller F. Language models based on Hebbian cell assemblies. J Neurosci (Paris). 2006;100:16–30.
Wickelgren WA. Context-sensitive coding, associative memory, and serial order in (speech) behavior. Psychol Rev. 1969;76:1–15.
Wickelgren WA. Webs, cell assemblies, and chunking in neural nets. Concepts Neurosci. 1992;3:1–53.
Wijekoon JHB, Dudek P. Compact silicon neuron circuit with spiking and bursting behaviour. Neural Netw. 2008;21:524–34.
Wille R. Restructuring lattice theory: an approach based on hierarchies of concepts. In: Rival I, editor. Ordered sets. Dordrecht: Reidel; 1982. p. 445–70.
Willshaw DJ, Buneman OP, Longuet-Higgins HC. Non-holographic associative memory. Nature. 1969;222:960–2.
Wolff KE. A first course in formal concept analysis. In: Faulbaum F, editors. StatSoft ’93 Advances in statistical software. Gustav Fischer Verlag; 1993. p. 429–38.
Yamashita Y, Tani J. Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment. PLoS Comput Biol 2008;4:e1000220
Acknowledgement
The author acknowledges support by the Engineering and Physical Sciences Research Council (grant EP/C010841/1, COLAMN—A Novel Computing Architecture for Cognitive Systems based on the Laminar Microcircuitry of the Neocortex) and the European Community (FACETS—Fast Analog Computing with Emergent Transient States). Comments by the reviewer are also greatly acknowledged.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wennekers, T. On the Natural Hierarchical Composition of Cliques in Cell Assemblies. Cogn Comput 1, 128–138 (2009). https://doi.org/10.1007/s12559-008-9004-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12559-008-9004-5