Abstract
In agreement with Bond’s suggestion, we consider that episodic memories are hierarchized autonomously by simple rule. In this research, our model solves maze tasks. Each episodic memory corresponds to the model’s each track. In our previous research, we suggested that our model concatenates episodic memories into one long episodic memory. Our previous model showed successful prediction of any long periodical and deterministic environmental changes with editing (selecting and concatenating with adequate timing) stored episodic memories autonomously. However, the previous models could not select adequate actions under a stochastic environment like POMDPs. Here, we suggest hierarchical episodic memories implement into the model. It is shown that the model improved not only their action under POMDPs but also prediction of long-term environmental change and incremental learning.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Tulving, E.: Elements of episodic memory. Oxford University Press, Oxford (1983)
Bond, A.H.: Representing episodic memory in a system-level model of the brain. Neurocomputing 65-66, 261–273 (2005)
Eichenbaum, H., Dudchenko, P., Wood, E., Shapiro, M., Tanila, H.: The hippocampus, memory, review and place cells: is it spatial memory or a memory space? Neuron. 23, 209–226 (1999)
Aota, Y., Miyake, Y., Ukai, S.: Neural network modeling of hippocampal place cells in rats. IEICE Transactions on Information and Systems J82-DII, 2355–2366 (1999)
Shapiro, M.L., Tanila, H., Eichenbaum, H.: Cues that hippocampal place cells encode: dynamic and hierarchical representation of local and distal stimuli. Hippocampus 7, 624–642 (1997)
Skaggs, W.E., McNaughton, B.L.: Spatial firing properties of hippocampal CA1 populations in an environment containing two visually identical regions. J. Neurosci. 18(20), 8455–8466 (1998)
Smith, D.M., Mizumori, S.J.Y.: Hippocampal place cells, context, and episodic memory. Hippocampus 16, 716–729 (2006)
Lipton, P.A., White, J.A., Eichenbaum, H.: Disambiguation of Overlapping Experiences by Neurons in the Medial Entorhinal Cortex. J. Neurosci. 27(21), 5787–5795 (2007)
Atance, C.M., O’Neill, D.K.: Episodic future thinking. Trends. Cogn. Sci. 5(12), 533–539 (2001)
Aota, Y., Yamaguchi, Y.: An algorithm for solving maze tasks with episodic memory integration. IPSJ Trans. Math. Model. Apps. 47(14), 93–107 (2006)
Aota, Y., Yamaguchi, Y.: Relationship between structure of tasks and past experiences on the learning algorithm with episodic memory integration. IPSJ Trans. Math. Model. Apps. 48(6), 12–22 (2007)
Zilli, E.A., Hasselmo, M.E.: Modeling the role of working memory and episodic memory in behavioral tasks. Hippocampus 18, 193–209 (2008)
Wickramasinghe, L.K., Alahakoon, L.D., Smith-Miles, K.: A novel episodic associative memory model for enhanced classification accuracy. Pattern Recognition Letters 28, 1193–1202 (2007)
Nuxoll, A., Laird, J.E.: Enhancing intelligent agents with episodic memory. Cogn. Sys. Res. 17-18, 34–48 (2012)
Brom, C., Lukavsky, J.: Towards virtual characters with a full episodic memory II: The episodic memory strikes back. In: Proc. Empathic Agents, AAMAS Workshop, pp. 1–9 (2009)
Miyazaki, K., Kobayashi, S.: Learning deterministic policies in partially observable markov decision processes. In: Proceedings of the 5th International Conference on Intelligent Autonomous System (IAS-5), pp. 250–257 (1998)
Laird, J.E.: Extending the Soar cognitive architecture. In: Proceedings of the First Artificial General Intelligence Conference, pp. 224–235 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Aota, Y., Miyake, Y. (2012). Future Prediction with Hierarchical Episodic Memories under Deterministic and Stochastic Environments. In: Huang, T., Zeng, Z., Li, C., Leung, C.S. (eds) Neural Information Processing. ICONIP 2012. Lecture Notes in Computer Science, vol 7663. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34475-6_31
Download citation
DOI: https://doi.org/10.1007/978-3-642-34475-6_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34474-9
Online ISBN: 978-3-642-34475-6
eBook Packages: Computer ScienceComputer Science (R0)