Abstract
The hippocampus plays a central role in spatial representation, declarative and episodic memory. In this area, so-called place cells possess high spatial selectivity, firing preferentially when the individual is within a small area of the environment. Interestingly, it has been found in rats that these cells can be active also when the animal is outside the location or context of their corresponding place field producing so-called “forward sweeps”. These typically occur at decision points during task execution and seem to be utilized, among other things, for the evaluation of potential alternative paths. Anticipatory firing is also found in the ventral striatum, a brain area that is strongly interconnected with the hippocampus and is known to encode value and reward. In this paper, we describe a biologically based computational model of the hippocampal-ventral striatum circuit that implements a goal-directed mechanism of choice, with the hippocampus primarily involved in the mental simulation of possible navigation paths and the ventral striatum involved in the evaluation of the associated reward expectancies. The model is validated in a navigation task in which a rat is placed in a complex maze with multiple rewarding sites. We show that the rat mentally activates place cells to simulate paths, estimate their value, and make decisions, implementing two essential processes of model-based reinforcement learning algorithms of choice: look-ahead prediction and the evaluation of predicted states.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Buckner RL, Carroll DC (2007) Self-projection and the brain. Trends Cogn Sci 11:49–57
Dayan P, Abbott LF (2001) Theoretical neuroscience. Computational and mathematical modelling of neural systems. MIT Press, Cambridge
Dragoi G, Tonegawa S (2011) Preplay of future place cell sequences by hippocampal cellular assemblies. Nature 469:397–401
Hassabis D, Kumaran D, Maguire EA (2007) Using imagination to understand the neural basis of episodic memory. J Neurosci 27:14365–14374
Johnson A, Redish AD (2007) Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J Neurosci 27:12176–12189
Lansink CS, Goltstein PM, Lankelma JV, McNaughton BL, Pennartz CMA (2009) Hippocampus leads ventral striatum in replay of place-reward information. PLoS Biol 7:e1000173
Lei G (1990) A neuron model with fluid properties for solving labyrinthian puzzle. Biol Cybern 64:61–67
Leutgeb S, Leutgeb JK, Barnes CA, Moser EI, McNaughton BL, Moser M-B (2005) Independent codes for spatial and episodic memory in hippocampal neuronal ensembles. Science 309:619–623
Leutgeb JK, Leutgeb S, Moser M-B, Moser EI (2007) Pattern separation in the dentate gyrus and CA3 of the hippocampus. Science 315:961–966
Lisman J, Redish AD (2009) Prediction, sequences and the hippocampus. Phil Trans R Soc B 364:1193–1201
Martinet L-E, Sheynikhovich D, Benchenane K, Arleo A (2011) Spatial learning and action planning in a prefrontal cortical network model. PLoS Comput Biol 7:e1002045
Pennartz CMA, Ito R, Verschure PFMJ, Battaglia FP, Robbins TW (2011) The hippocampal-striatal axis in learning, prediction and goal-directed behavior. Trends Neurosci 34:548–559
Pezzulo G, Rigoli F, Chersi F (2012) A mixed instrumental controller can combine habitual and goal-directed choice (submitted)
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge
Conflict of interest
This supplement was not sponsored by outside commercial interests. It was funded entirely by ECONA, Via dei Marsi, 78, 00185 Roma, Italy.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chersi, F., Pezzulo, G. Using hippocampal-striatal loops for spatial navigation and goal-directed decision-making. Cogn Process 13 (Suppl 1), 125–129 (2012). https://doi.org/10.1007/s10339-012-0475-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10339-012-0475-7