Nothing Special   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/1009634.html
   My bibliography  Save this article

Optimism and pessimism in optimised replay

Author

Listed:
  • Georgy Antonov
  • Christopher Gagne
  • Eran Eldar
  • Peter Dayan
Abstract
The replay of task-relevant trajectories is known to contribute to memory consolidation and improved task performance. A wide variety of experimental data show that the content of replayed sequences is highly specific and can be modulated by reward as well as other prominent task variables. However, the rules governing the choice of sequences to be replayed still remain poorly understood. One recent theoretical suggestion is that the prioritization of replay experiences in decision-making problems is based on their effect on the choice of action. We show that this implies that subjects should replay sub-optimal actions that they dysfunctionally choose rather than optimal ones, when, by being forgetful, they experience large amounts of uncertainty in their internal models of the world. We use this to account for recent experimental data demonstrating exactly pessimal replay, fitting model parameters to the individual subjects’ choices.Author summary: When animals are asleep or restfully awake, populations of neurons in their brains recapitulate activity associated with extended behaviourally-relevant experiences. This process is called replay, and it has been established for a long time in rodents, and very recently in humans, to be important for good performance in decision-making tasks. The specific experiences which are replayed during those epochs follow highly ordered patterns, but the mechanisms which establish their priority are still not fully understood. One promising theoretical suggestion is that each replay experience is chosen in such a way that the learning that ensues is most helpful for the subsequent performance of the animal. A very recent study reported a surprising result that humans who achieved high performance in a planning task tended to replay actions they found to be sub-optimal, and that this was associated with a useful deprecation of those actions in subsequent performance. In this study, we examine the nature of this pessimized form of replay and show that it is exactly appropriate for forgetful agents. We analyse the role of forgetting for replay choices of our model, and verify our predictions using human subject data.

Suggested Citation

  • Georgy Antonov & Christopher Gagne & Eran Eldar & Peter Dayan, 2022. "Optimism and pessimism in optimised replay," PLOS Computational Biology, Public Library of Science, vol. 18(1), pages 1-32, January.
    • Handle: RePEc:plo:pcbi00:1009634
    DOI: 10.1371/journal.pcbi.1009634
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1009634
    Download Restriction: no
    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1009634&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pcbi.1009634?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. George Dragoi & Susumu Tonegawa, 2011. "Preplay of future place cell sequences by hippocampal cellular assemblies," Nature, Nature, vol. 469(7330), pages 397-401, January.
    2. Brad E. Pfeiffer & David J. Foster, 2013. "Hippocampal place-cell sequences depict future paths to remembered goals," Nature, Nature, vol. 497(7447), pages 74-79, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Will D Penny & Peter Zeidman & Neil Burgess, 2013. "Forward and Backward Inference in Spatial Cognition," PLOS Computational Biology, Public Library of Science, vol. 9(12), pages 1-22, December.
    2. Lukas Grossberger & Francesco P Battaglia & Martin Vinck, 2018. "Unsupervised clustering of temporal patterns in high-dimensional neuronal ensembles using a novel dissimilarity measure," PLOS Computational Biology, Public Library of Science, vol. 14(7), pages 1-34, July.
    3. J Matthew Mahoney & Ali S Titiz & Amanda E Hernan & Rod C Scott, 2016. "Short-Range Temporal Interactions in Sleep; Hippocampal Spike Avalanches Support a Large Milieu of Sequential Activity Including Replay," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-25, February.
    4. Marta Huelin Gorriz & Masahiro Takigawa & Daniel Bendor, 2023. "The role of experience in prioritizing hippocampal replay," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    5. Nicolas Cazin & Martin Llofriu Alonso & Pablo Scleidorovich Chiodi & Tatiana Pelc & Bruce Harland & Alfredo Weitzenfeld & Jean-Marc Fellous & Peter Ford Dominey, 2019. "Reservoir computing model of prefrontal cortex creates novel combinations of previous navigation sequences from hippocampal place-cell replay with spatial reward propagation," PLOS Computational Biology, Public Library of Science, vol. 15(7), pages 1-32, July.
    6. Hong Yu & Xinkuan Xiang & Zongming Chen & Xu Wang & Jiaqi Dai & Xinxin Wang & Pengcheng Huang & Zheng-dong Zhao & Wei L. Shen & Haohong Li, 2021. "Periaqueductal gray neurons encode the sequential motor program in hunting behavior of mice," Nature Communications, Nature, vol. 12(1), pages 1-15, December.
    7. Anli A. Liu & Simon Henin & Saman Abbaspoor & Anatol Bragin & Elizabeth A. Buffalo & Jordan S. Farrell & David J. Foster & Loren M. Frank & Tamara Gedankien & Jean Gotman & Jennifer A. Guidera & Kari , 2022. "A consensus statement on detection of hippocampal sharp wave ripples and differentiation from other fast oscillations," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    8. Hiroyuki Miyawaki & Kenji Mizuseki, 2022. "De novo inter-regional coactivations of preconfigured local ensembles support memory," Nature Communications, Nature, vol. 13(1), pages 1-21, December.
    9. Trygve Solstad & Hosam N Yousif & Terrence J Sejnowski, 2014. "Place Cell Rate Remapping by CA3 Recurrent Collaterals," PLOS Computational Biology, Public Library of Science, vol. 10(6), pages 1-10, June.
    10. Francesco Donnarumma & Domenico Maisto & Giovanni Pezzulo, 2016. "Problem Solving as Probabilistic Inference with Subgoaling: Explaining Human Successes and Pitfalls in the Tower of Hanoi," PLOS Computational Biology, Public Library of Science, vol. 12(4), pages 1-30, April.
    11. John Palmer & Adam Keane & Pulin Gong, 2017. "Learning and executing goal-directed choices by internally generated sequences in spiking neural circuits," PLOS Computational Biology, Public Library of Science, vol. 13(7), pages 1-23, July.
    12. Hefei Guan & Steven J. Middleton & Takafumi Inoue & Thomas J. McHugh, 2021. "Lateralization of CA1 assemblies in the absence of CA3 input," Nature Communications, Nature, vol. 12(1), pages 1-10, December.
    13. Alexander Nitsch & Mona M. Garvert & Jacob L. S. Bellmund & Nicolas W. Schuck & Christian F. Doeller, 2024. "Grid-like entorhinal representation of an abstract value space during prospective decision making," Nature Communications, Nature, vol. 15(1), pages 1-20, December.
    14. Yang Yiling & Katharine Shapcott & Alina Peter & Johanna Klon-Lipok & Huang Xuhui & Andreea Lazar & Wolf Singer, 2023. "Robust encoding of natural stimuli by neuronal response sequences in monkey visual cortex," Nature Communications, Nature, vol. 14(1), pages 1-18, December.
    15. Takamitsu Iwata & Takufumi Yanagisawa & Yuji Ikegaya & Jonathan Smallwood & Ryohei Fukuma & Satoru Oshino & Naoki Tani & Hui Ming Khoo & Haruhiko Kishima, 2024. "Hippocampal sharp-wave ripples correlate with periods of naturally occurring self-generated thoughts in humans," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    16. Guillaume Viejo & Thomas Cortier & Adrien Peyrache, 2018. "Brain-state invariant thalamo-cortical coordination revealed by non-linear encoders," PLOS Computational Biology, Public Library of Science, vol. 14(3), pages 1-25, March.
    17. Bahaaeddin Attaallah & Pierre Petitet & Rhea Zambellas & Sofia Toniolo & Maria Raquel Maio & Akke Ganse-Dumrath & Sarosh R. Irani & Sanjay G. Manohar & Masud Husain, 2024. "The role of the human hippocampus in decision-making under uncertainty," Nature Human Behaviour, Nature, vol. 8(7), pages 1366-1382, July.
    18. Alex P. Vaz & John H. Wittig & Sara K. Inati & Kareem A. Zaghloul, 2023. "Backbone spiking sequence as a basis for preplay, replay, and default states in human cortex," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    19. Ruy Gómez-Ocádiz & Massimiliano Trippa & Chun-Lei Zhang & Lorenzo Posani & Simona Cocco & Rémi Monasson & Christoph Schmidt-Hieber, 2022. "A synaptic signal for novelty processing in the hippocampus," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    20. Jordan Crivelli-Decker & Alex Clarke & Seongmin A. Park & Derek J. Huffman & Erie D. Boorman & Charan Ranganath, 2023. "Goal-oriented representations in the human hippocampus during planning and navigation," Nature Communications, Nature, vol. 14(1), pages 1-13, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1009634. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.