A model of autonomous interactions between hippocampus and neocortex driving sleep-dependent memory consolidation

Dhairyya Singh, Kenneth A Norman, Anna C Schapiro, Dhairyya Singh, Kenneth A Norman, Anna C Schapiro

Abstract

How do we build up our knowledge of the world over time? Many theories of memory formation and consolidation have posited that the hippocampus stores new information, then "teaches" this information to the neocortex over time, especially during sleep. But it is unclear, mechanistically, how this actually works-How are these systems able to interact during periods with virtually no environmental input to accomplish useful learning and shifts in representation? We provide a framework for thinking about this question, with neural network model simulations serving as demonstrations. The model is composed of hippocampus and neocortical areas, which replay memories and interact with one another completely autonomously during simulated sleep. Oscillations are leveraged to support error-driven learning that leads to useful changes in memory representation and behavior. The model has a non-rapid eye movement (NREM) sleep stage, where dynamics between the hippocampus and neocortex are tightly coupled, with the hippocampus helping neocortex to reinstate high-fidelity versions of new attractors, and a REM sleep stage, where neocortex is able to more freely explore existing attractors. We find that alternating between NREM and REM sleep stages, which alternately focuses the model's replay on recent and remote information, facilitates graceful continual learning. We thus provide an account of how the hippocampus and neocortex can interact without any external input during sleep to drive useful new cortical learning and to protect old knowledge as new information is integrated.

Keywords: continual learning; neural network model; oscillations; sleep stages.

Conflict of interest statement

The authors declare no competing interest.

Figures

Fig. 1.
Fig. 1.
Model architecture and sleep algorithm. (A) The architecture included C-HORSE (our model of the hippocampus; purple outlines), and a neocortical layer as the target of consolidation (orange outline). (B) When replaying attractor X, all units participating in the attractor have strong, stable activity during the plus phase. During the minus phase, oscillatory phases of higher inhibition lead to lower activity, with the weakest units in attractor X dropping out, and oscillatory phases of lower inhibition lead to higher activity, causing activity spreading to a nearby attractor, Y. (C) Stability trace for a real learning event, with background colors indicating the plus and minus phases. When the model first falls into an attractor, its activity is highly stable and triggers the plus phase. Short-term synaptic depression gradually destabilizes the attractor. The stability drop causes the plus phase to end, and the minus phase begins. As the attractor further destabilizes, the minus phase ends.
Fig. 2.
Fig. 2.
Schapiro et al. (29) category learning paradigm and results. (A) Satellite exemplars studied from three categories. (B) Performance change for unique and shared features from presleep to postsleep tests. *P < 0.05. (Results are collapsed across the frequency manipulation included in that study.) (C) Experimental protocol: Participants studied the satellites in the evening, had an immediate test, and were tested again 12 h later, after a night of sleep.
Fig. 3.
Fig. 3.
Simulation 1: Category learning consolidation across one night of sleep. (A) Simulation protocol. (B) Change in full model performance from before to after sleep for unique features, shared features, and cross-cutting features. (C) Chord plot of replay transitions, with A, B, and C indicating categories. Arrow base indicates transition initiation, and head indicates termination. Arrow width indicates proportion of replays for a given satellite. The plot shows that replay was interleaved uniformly across exemplars (with a modest preference for within-category transitions). (D) (Left) Change in pairwise correlation of neocortical representations across items from before to after sleep. Black boxes indicate within-category changes. (Right) Averaged pairwise correlation change of neocortical representations within and across categories. (E) Fisher-transformed correlation between exemplar replay frequency during sleep and performance change, averaged across model initializations. All error bars represent ±1 SEM across network initializations.
Fig. 4.
Fig. 4.
Simulation 2: Continual learning via alternating sleep stages. (A) Simulation protocol. (B) L1 distance of plus phase attractors to Env 1 and Env 2 item patterns in REM and NREM. Smaller distance indicates more replay of those patterns. (C) Mean performance across sleep for Env 1 and Env 2 items for NREM-only vs. alternating NREM/REM (N/R) vs. REM-only conditions.

References

    1. McClelland J. L., McNaughton B. L., O’Reilly R. C., Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychol. Rev. 102, 419–457 (1995).
    1. Diekelmann S., Born J., The memory function of sleep. Nat. Rev. Neurosci. 11, 114–126 (2010).
    1. Rasch B., Born J., About sleep’s role in memory. Physiol. Rev. 93, 681–766 (2013).
    1. Klinzing J. G., Niethard N., Born J., Mechanisms of systems memory consolidation during sleep. Nat. Neurosci. 22, 1598–1610 (2019).
    1. Geva-Sagiv M., Nir Y., Local sleep oscillations: implications for memory consolidation. Front. Neurosci. 13, 813 (2019).
    1. Tamminen J., Payne J. D., Stickgold R., Wamsley E. J., Gaskell M. G., Sleep spindle activity is associated with the integration of new memories and existing knowledge. J. Neurosci. 30, 14356–14360 (2010).
    1. Tamminen J., Lambon Ralph M. A., Lewis P. A., The role of sleep spindles and slow-wave activity in integrating new information in semantic memory. J. Neurosci. 33, 15376–15381 (2013).
    1. Xia F., et al. , Parvalbumin-positive interneurons mediate neocortical-hippocampal interactions that are necessary for memory consolidation. eLife 6, e27868 (2017).
    1. Khodagholy D., Gelinas J. N., Buzsáki G., Learning-enhanced coupling between ripple oscillations in association cortices and hippocampus. Science 358, 369–372 (2017).
    1. Logothetis N. K., et al. , Hippocampal-cortical interaction during periods of subcortical silence. Nature 491, 547–553 (2012).
    1. Ji D., Wilson M. A., Coordinated memory replay in the visual cortex and hippocampus during sleep. Nat. Neurosci. 10, 100–107 (2007).
    1. Ngo H.-V. V., Martinetz T., Born J., Mölle M., Auditory closed-loop stimulation of the sleep slow oscillation enhances memory. Neuron 78, 545–553 (2013).
    1. Maingret N., Girardeau G., Todorova R., Goutierre M., Zugaro M., Hippocampo-cortical coupling mediates memory consolidation during sleep. Nat. Neurosci. 19, 959–964 (2016).
    1. Ladenbauer J., et al. , Promoting sleep oscillations and their functional coupling by transcranial stimulation enhances memory consolidation in mild cognitive impairment. J. Neurosci. 37, 7111–7124 (2017).
    1. Geva-Sagiv M., et al. , Hippocampal-prefrontal neuronal synchrony during human sleep mediates memory consolidation (Presented at Society for Neuroscience, 2021) [Poster]. . Accessed 1 December 2021.
    1. Ketz N., Jones A. P., Bryant N. B., Clark V. P., Pilly P. K., Closed-loop slow-wave tACS improves sleep-dependent long-term memory generalization by modulating endogenous oscillations. J. Neurosci. 38, 7314–7326 (2018).
    1. Squire L. R., Alvarez P., Retrograde amnesia and memory consolidation: A neurobiological perspective. Curr. Opin. Neurobiol. 5, 169–177 (1995).
    1. Winocur G., Moscovitch M., Sekeres M., Memory consolidation or transformation: Context manipulation and hippocampal representations of memory. Nat. Neurosci. 10, 555–557 (2007).
    1. Nadel L., Moscovitch M., Memory consolidation, retrograde amnesia and the hippocampal complex. Curr. Opin. Neurobiol. 7, 217–227 (1997).
    1. Winocur G., Moscovitch M., Memory transformation and systems consolidation. J. Int. Neuropsychol. Soc. 17, 766–780 (2011).
    1. Moscovitch M., Gilboa A., Systems consolidation, transformation and reorganization: Multiple trace theory, trace transformation theory and their competitors. PsyArXiv [Preprint] (2021). 10.31234/. Accessed 4 December 2021.
    1. Yonelinas A. P., Ranganath C., Ekstrom A. D., Wiltgen B. J., A contextual binding theory of episodic memory: Systems consolidation reconsidered. Nat. Rev. Neurosci. 20, 364–375 (2019).
    1. Kitamura T., et al. , Engrams and circuits crucial for systems consolidation of a memory. Science 356, 73–78 (2017).
    1. Antony J. W., Schapiro A. C., Active and effective replay: Systems consolidation reconsidered again. Nat. Rev. Neurosci. 20, 506–507 (2019).
    1. Kowadlo G., Ahmed A., Rawlinson D., One-shot learning for the long term: Consolidation with an artificial hippocampal algorithm. ArXiv [Preprint] (2021). 10.48550/arXiv.2102.07503. Accessed 10 May 2022.
    1. Norman K. A., Newman E. L., Perotte A. J., Methods for reducing interference in the Complementary Learning Systems model: Oscillating inhibition and autonomous memory rehearsal. Neural Netw. 18, 1212–1228 (2005).
    1. Goto A., et al. , Stepwise synaptic plasticity events drive the early phase of memory consolidation. Science 374, 857–863 (2021).
    1. Friedrich M., Wilhelm I., Born J., Friederici A. D., Generalization of word meanings during infant sleep. Nat. Commun. 6, 6004 (2015).
    1. Schapiro A. C., et al. , Sleep benefits memory for semantic category structure while preserving exemplar-specific information. Sci. Rep. 7, 14869 (2017).
    1. Huguet M., Payne J. D., Kim S. Y., Alger S. E., Overnight sleep benefits both neutral and negative direct associative and relational memory. Cogn. Affect. Behav. Neurosci. 19, 1391–1403 (2019).
    1. Schapiro A. C., Turk-Browne N. B., Botvinick M. M., Norman K. A., Complementary learning systems within the hippocampus: A neural network modelling approach to reconciling episodic memory with statistical learning. Philos. Trans. R. Soc. Lond. B Biol. Sci. 372, 20160049 (2017).
    1. Zhou Z., Singh D., Tandoc M. C., Schapiro A. C., Distributed representations for human inference. bioRxiv [Preprint] (2021). 10.1101/2021.07.29.454337. Accessed 1 December 2021.
    1. Sučević J., Schapiro A. C., A neural network model of hippocampal contributions to category learning. bioRxiv [Preprint] (2022). 10.1101/2022.01.12.476051. Accessed 18 January 2022.
    1. O’Reilly R. C., Munakata Y., Frank M. J., Hazy T. E., Computational cognitive neuroscience, ed. 4. . Accessed 1 December 2021.
    1. Niethard N., Ngo H.-V. V., Ehrlich I., Born J., Cortical circuit activity underlying sleep slow oscillations and spindles. Proc. Natl. Acad. Sci. U.S.A. 115, E9220–E9229 (2018).
    1. Durkin J., et al. , Cortically coordinated NREM thalamocortical oscillations play an essential, instructive role in visual system plasticity. Proc. Natl. Acad. Sci. U.S.A. 114, 10485–10490 (2017).
    1. Peyrache A., Seibt J., A mechanism for learning with sleep spindles. Philos. Trans. R. Soc. Lond. B Biol. Sci. 375, 20190230 (2020).
    1. Boyce R., Glasgow S. D., Williams S., Adamantidis A., Causal evidence for the role of REM sleep theta rhythm in contextual memory consolidation. Science 352, 812–816 (2016).
    1. Poe G. R., Nitz D. A., McNaughton B. L., Barnes C. A., Experience-dependent phase-reversal of hippocampal neuron firing during REM sleep. Brain Res. 855, 176–180 (2000).
    1. Treves A., Frontal latching networks: A possible neural basis for infinite recursion. Cogn. Neuropsychol. 22, 276–291 (2005).
    1. Ackley D. H., Hinton G. E., Sejnowski T. J., A learning algorithm for Boltzmann machines. Cogn. Sci. 9, 147–169 (1985).
    1. Schapiro A. C., McDevitt E. A., Rogers T. T., Mednick S. C., Norman K. A., Human hippocampal replay during rest prioritizes weakly learned information and predicts memory performance. Nat. Commun. 9, 3920 (2018).
    1. Ralph M. A. L., Jefferies E., Patterson K., Rogers T. T., The neural and computational bases of semantic cognition. Nat. Rev. Neurosci. 18, 42–55 (2017).
    1. van Kesteren M. T. R., Ruiter D. J., Fernández G., Henson R. N., How schema and novelty augment memory formation. Trends Neurosci. 35, 211–219 (2012).
    1. Tompary A., Davachi L., Consolidation promotes the emergence of representational overlap in the hippocampus and medial prefrontal cortex. Neuron 96, 228–241.e5 (2017).
    1. Sun L., Zhou H., Cichon J., Yang G., Experience and sleep-dependent synaptic plasticity: From structure to activity. Philos. Trans. R. Soc. Lond. B Biol. Sci. 375, 20190234 (2020).
    1. McCloskey M., Cohen N. J., “Catastrophic interference in connectionist networks: The sequential learning problem” in Psychology of Learning and Motivation, Bower G. H., Ed. (Elsevier, 1989), vol. 24, pp. 109–165.
    1. Barnes J. M., Underwood B. J., Fate of first-list associations in transfer theory. J. Exp. Psychol. 58, 97–105 (1959).
    1. Drosopoulos S., Schulze C., Fischer S., Born J., Sleep’s function in the spontaneous recovery and consolidation of memories. J. Exp. Psychol. Gen. 136, 169–183 (2007).
    1. Ellenbogen J. M., Hulbert J. C., Stickgold R., Dinges D. F., Thompson-Schill S. L., Interfering with theories of sleep and memory: Sleep, declarative memory, and associative interference. Curr. Biol. 16, 1290–1294 (2006).
    1. Baran B., Wilson J., Spencer R. M. C., REM-dependent repair of competitive memory suppression. Exp. Brain Res. 203, 471–477 (2010).
    1. McClelland J. L., McNaughton B. L., Lampinen A. K., Integration of new information in memory: New insights from a complementary learning systems perspective. Philos. Trans. R. Soc. Lond. B Biol. Sci. 375, 20190637 (2020).
    1. Hayes T. L., et al. , Replay in deep learning: Current approaches and missing biological elements. ArXiv [Preprint] (2021). 10.48550/arXiv.2104.04132. Accessed 28 January 2022.
    1. González O. C., Sokolov Y., Krishnan G. P., Delanois J. E., Bazhenov M., Can sleep protect memories from catastrophic forgetting? eLife 9, e51005 (2020).
    1. Sanda P., et al. , Bidirectional interaction of hippocampal ripples and cortical slow waves leads to coordinated spiking activity during NREM sleep. Cereb. Cortex 31, 324–340 (2021).
    1. Wei Y., Krishnan G. P., Komarov M., Bazhenov M., Differential roles of sleep spindles and sleep slow oscillations in memory consolidation. PLOS Comput. Biol. 14, e1006322 (2018).
    1. Welling M., Hinton G. E., “A new learning algorithm for mean field Boltzmann machines” in Artificial Neural Networks — ICANN 2002, Dorronsoro R., Ed. (Lecture Notes in Computer Science, Springer, 2002), vol. J, pp. 351–357.
    1. Scellier B., Bengio Y., Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Front. Comput. Neurosci. 11, 24 (2017).
    1. O’Reilly R. C., Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm. Neural Comput. 8, 895–938 (1996).
    1. Girardeau G., Benchenane K., Wiener S. I., Buzsáki G., Zugaro M. B., Selective suppression of hippocampal ripples impairs spatial memory. Nat. Neurosci. 12, 1222–1223 (2009).
    1. Stickgold R., Walker M. P., Sleep-dependent memory triage: Evolving generalization through selective processing. Nat. Neurosci. 16, 139–145 (2013).
    1. Landmann N., et al. , The reorganisation of memory during sleep. Sleep Med. Rev. 18, 531–541 (2014).
    1. Lewis P. A., Durrant S. J., Overlapping memory replay during sleep builds cognitive schemata. Trends Cogn. Sci. 15, 343–351 (2011).
    1. Mirković J., Gaskell M. G., Does sleep improve your grammar? Preferential consolidation of arbitrary components of new linguistic knowledge. PLoS One 11, e0152489 (2016).
    1. Hanert A., Weber F. D., Pedersen A., Born J., Bartsch T., Sleep in humans stabilizes pattern separation performance. J. Neurosci. 37, 12238–12246 (2017).
    1. D. J. Cai, S. A. Mednick, E. M. Harrison, J. C. Kanady, S. C. Mednick, REM, not incubation, improves creativity by priming associative networks. Proceedings of the National Academy of Sciences 106, 10130–10134 (2009).
    1. Batterink L. J., Westerberg C. E., Paller K. A., Vocabulary learning benefits from REM after slow-wave sleep. Neurobiol. Learn. Mem. 144, 102–113 (2017).
    1. Tamminen J., Lambon Ralph M. A., Lewis P. A., Targeted memory reactivation of newly learned words during sleep triggers REM-mediated integration of new memories and existing knowledge. Neurobiol. Learn. Mem. 137, 77–82 (2017).
    1. Giuditta A., et al. , The sequential hypothesis of the function of sleep. Behav. Brain Res. 69, 157–166 (1995).
    1. Strauss M., et al. , Order matters: Sleep spindles contribute to memory consolidation only when followed by rapid-eye-movement sleep. Sleep 45, zsac022 (2022).
    1. McDevitt E. A., Duggan K. A., Mednick S. C., REM sleep rescues learning from interference. Neurobiol. Learn. Mem. 122, 51–62 (2015).
    1. Li Z., Hoiem D., Learning without forgetting. arXiv [Preprint] (2016). . Accessed 28 January 2022.
    1. Shin H., Lee J. K., Kim J., Kim J., Continual learning with deep generative replay. arXiv [Preprint] (2017). . Accessed 30 January 2022.
    1. van de Ven G. M., Siegelmann H. T., Tolias A. S., Brain-inspired replay for continual learning with artificial neural networks. Nat. Commun. 11, 4069 (2020).
    1. Rebuffi S.-A., Kolesnikov A., Sperl G., Lampert C. H., iCaRL: Incremental classifier and representation learning. arXiv [Preprint] (2017). . Accessed 30 January 2022.
    1. Gepperth A., Karaoguz C., A bio-inspired incremental learning architecture for applied perceptual problems. Cognit. Comput. 8, 924–934 (2016).
    1. Tononi G., Cirelli C., Sleep and the price of plasticity: From synaptic and cellular homeostasis to memory consolidation and integration. Neuron 81, 12–34 (2014).
    1. Sawangjit A., et al. , The hippocampus is crucial for forming non-hippocampal long-term memory during sleep. Nature 564, 109–113 (2018).
    1. Schapiro A. C., et al. , The hippocampus is necessary for the consolidation of a task that does not require the hippocampus for initial learning. Hippocampus 29, 1091–1100 (2019).
    1. Wamsley E. J., Stickgold R., Memory, sleep and dreaming: Experiencing consolidation. Sleep Med. Clin. 6, 97–108 (2011).
    1. Louie K., Wilson M. A., Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep. Neuron 29, 145–156 (2001).
    1. Schönauer M., et al. , Decoding material-specific memory reprocessing during sleep in humans. Nat. Commun. 8, 15404 (2017).
    1. Tambini A., Davachi L., Persistence of hippocampal multivoxel patterns into postencoding rest is related to memory. Proc. Natl. Acad. Sci. U.S.A. 110, 19591–19596 (2013).
    1. Tambini A., Davachi L., Awake reactivation of prior experiences consolidates memories and biases cognition. Trends Cogn. Sci. 23, 876–890 (2019).
    1. Schapiro A. C., Rogers T. T., Cordova N. I., Turk-Browne N. B., Botvinick M. M., Neural representations of events arise from temporal community structure. Nat. Neurosci. 16, 486–492 (2013).
    1. Jadhav S. P., Kemere C., German P. W., Frank L. M., Awake hippocampal sharp-wave ripples support spatial memory. Science 336, 1454–1458 (2012).
    1. Squire L. R., Genzel L., Wixted J. T., Morris R. G., Memory consolidation. Cold Spring Harb. Perspect. Biol. 7, a021766 (2015).
    1. Mattar M. G., Daw N. D., Prioritized memory access explains planning and hippocampal replay. Nat. Neurosci. 21, 1609–1617 (2018).
    1. Liu X., Kuzum D., Hippocampal-cortical memory trace transfer and reactivation through cell-specific stimulus and spontaneous background noise. Front. Comput. Neurosci. 13, 67 (2019).
    1. Tomé D. F., Sadeh S., Clopath C., Coordinated hippocampal-thalamic-cortical communication crucial for engram dynamics underneath systems consolidation. Nat. Commun. 13, 840 (2022).
    1. Sun W., Advani M., Spruston N., Saxe A., Fitzgerald J. E., Organizing memories for generalization in complementary learning systems. bioRxiv [Preprint] (2021). . Accessed 29 January 2022.
    1. Santoro A., Frankland P. W., Richards B. A., Memory transformation enhances reinforcement learning in dynamic environments. J. Neurosci. 36, 12228–12242 (2016).
    1. Cowan E. T., Schapiro A. C., Dunsmoor J. E., Murty V. P., Memory consolidation as an adaptive process. Psychon. Bull. Rev. 28, 1796–1810 (2021).
    1. Denis D., et al. , The roles of item exposure and visualization success in the consolidation of memories across wake and sleep. Learn. Mem. 27, 451–456 (2020).
    1. Gupta A. S., van der Meer M. A. A., Touretzky D. S., Redish A. D., Hippocampal replay is not a simple function of experience. Neuron 65, 695–705 (2010).
    1. Liu Y., Dolan R. J., Kurth-Nelson Z., Behrens T. E. J., Human replay spontaneously reorganizes experience. Cell 178, 640–652.e14 (2019).
    1. Kumaran D., McClelland J. L., Generalization through the recurrent interaction of episodic memories: A model of the hippocampal system. Psychol. Rev. 119, 573–616 (2012).
    1. Roxin A., Fusi S., Efficient partitioning of memory systems and its importance for memory consolidation. PLOS Comput. Biol. 9, e1003146 (2013).
    1. Himmer L., Schönauer M., Heib D. P. J., Schabus M., Gais S., Rehearsal initiates systems memory consolidation, sleep makes it last. Sci. Adv. 5, eaav1695 (2019).
    1. Lesburguères E., et al. , Early tagging of cortical networks is required for the formation of enduring associative memory. Science 331, 924–928 (2011).
    1. Tse D., et al. , Schemas and memory consolidation. Science 316, 76–82 (2007).
    1. McClelland J. L., Incorporating rapid neocortical learning of new schema-consistent information into complementary learning systems theory. J. Exp. Psychol. Gen. 142, 1190–1210 (2013).
    1. Vargha-Khadem F., et al. , Differential effects of early hippocampal pathology on episodic and semantic memory. Science 277, 376–380 (1997).
    1. Aisa B., Mingus B., O’Reilly R., The emergent neural modeling system. Neural Netw. 21, 1146–1152 (2008).
    1. Xie X., Seung H. S., Equivalence of backpropagation and contrastive Hebbian learning in a layered network. Neural Comput. 15, 441–454 (2003).
    1. Rothschild G., Eban E., Frank L. M., A cortical-hippocampal-cortical loop of information processing during memory consolidation. Nat. Neurosci. 20, 251–259 (2017).
    1. Hasselmo M. E., Neuromodulation: Acetylcholine and memory consolidation. Trends Cogn. Sci. 3, 351–359 (1999).
    1. Hartmann E., The 90-minute sleep-dream cycle. Arch. Gen. Psychiatry 18, 280–286 (1968).
    1. Singh D., Norman K. A., Schapiro A. C., A model of autonomous interactions between hippocampus and neocortex driving sleep-dependent memory consolidation. GitHub. . Deposited 30 August 2022.

Source: PubMed

Sponsors et collaborateurs

Les conditions médicales

Interventions en matière de drogue

3
S'abonner