Abstract
We compare the performance of drive- versus perception-based motivational systems in an unstable environment. We investigate the hypothesis that valence systems (systems that evaluate positive and negative nature of events) that are based on internal physiology will have an advantage over systems that are based purely on external sensory input. Results show that inclusion of internal drive levels in valence system input significantly improves performance. Furthermore, a valence system based purely on internal drives outperforms a system that is additionally based on perceptual input. We provide arguments for why this is so and relate our architecture to brain areas involved in animal learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Schultz, W., Dayan, P., Montague, P.R.: A neural substrate of prediction and reward. Science 275, 1593–1599 (1997)
Ackley, D.E., Littman, M.E.: Interactions between learning and evolution. In: Langton, C., Taylor, C., Farmer, D., Rasmussen, S. (eds.) Proc. Second Conf. Artificial Life (1991)
Singh, S., Barto, A.G., Chentanez, N.: Intrinsically Motivated Reinforcement Learning. In: Adv. Neural Information Processing Systems, vol. 17. MIT Press, Cambridge (2004)
Schembri, M., Mirolli, M., Baldassarre, G.: Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot. In: Demiris, Y., Mareschal, D., Scassellati, B., Weng, J. (eds.) Proc. 6th Int. Conf. Development and Learning, pp. E1–6. Imperial College, London (2006)
Damoulas, T., Cos-Aguilera, I., Hayes, G.M., Taylor, T.: Valency for adaptive homeostatic agents: Relating evolution and learning. In: Capcarrère, M.S., Freitas, A.A., Bentley, P.J., Johnson, C.G., Timmis, J. (eds.) ECAL 2005. LNCS (LNAI), vol. 3630, pp. 936–945. Springer, Berlin (2005)
Snel, M.: Evolving a sense of valency. MSc thesis, School of Informatics, University of Edinburgh (2007)
Cañamero, D.: Modeling motivations and emotions as a basis for intelligent behavior. In: Proc. Int. Conf. Autonomous Agents, pp. 148–155 (1997)
Velásquez, J.: Modeling emotion-based decision making. In: Cañamero, L.D. (ed.) Emotional and intelligent: The tangled of knot of cognition. Papers from the 1998 AAAI Fall Symposium, pp. 164–169. AAAI Press, Menlo Park (1998)
Cos-Aguilera, I., Cañamero, L., Hayes, G.M.: Motivation-driven learning of object affordances: first experiments using a simulated Khepera robot. In: Detjer, F., Dörner, D., Schaub, H. (eds.) The logic of cognitive systems: Proc. Fifth Int. Conf. Cognitive Modeling (ICCM 2003), pp. 57–62 (2003)
Urzelai, J., Floreano, D.: Evolution of adaptive synapses: robots with fast adaptive behavior in new environments. Evolutionary Computation 9(4), 495–524 (2001)
Capi, G., Doya, K.: Evolution of neural architecture fitting environmental dynamics. Adaptive Behavior 13(1), 53–66 (2005)
Barto, A.G., Sutton, R.S., Anderson, C.W.: Neuronlike elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics 13, 835–846 (1983)
Ackley, D.E., Littman, M.E.: Generalization and scaling in reinforcement learning. In: Touretzky, D.S. (ed.) Adv. Neural Information Processing Systems, vol. 2. Morgan Kaufmann, San Mateo (1990)
Sutton, R.S.: Learning to predict by the method of temporal differences. Machine Learning 3, 9–44 (1988)
Mayley, G.: Landscapes, learning costs and genetic assimilation. Evolutionary Computation 4(3), 213–234 (1996b)
Anderson, R.W.: Learning and evolution: a quantitative genetics approach. J. Theor. Biol. 175, 89–101 (1995)
Houk, J.C., Adams, J.L., Barto, A.G.: A model of how the basal ganglia generates and uses neural signals that predict reinforcement. In: Houk, J.C., Davis, J.L., Beiser, D.G. (eds.) Models of Information Processing in the Basal Ganglia, pp. 249–274. MIT Press, Cambridge (1995)
Olds, J.: Drives and reinforcements: behavioral studies of hypothalamic functions. Raven Press, New York (1977)
Conover, K.L., Woodside, B., Shizgal, P.: Effects of sodium depletion on competition and summation between rewarding effects of salt and lateral hypothalamic stimulation in the rat. Behav. Neurosci. 108, 549–558 (1994)
Valenstein, E.S., Cox, V.C., Kakolewski, J.W.: Modification of motivated behavior elicited by electrical stimulation of the hypothalamus. Science 157, 552–554 (1968)
Barto, A.G.: Adaptive critics and the basal ganglia. In: Houk, J.C., Davis, J.L., Beiser, D.G. (eds.) Models of Information Processing in the Basal Ganglia, pp. 215–232. MIT Press, Cambridge (1995)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Snel, M., Hayes, G.M. (2008). Evolution of Valence Systems in an Unstable Environment. In: Asada, M., Hallam, J.C.T., Meyer, JA., Tani, J. (eds) From Animals to Animats 10. SAB 2008. Lecture Notes in Computer Science(), vol 5040. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69134-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-69134-1_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69133-4
Online ISBN: 978-3-540-69134-1
eBook Packages: Computer ScienceComputer Science (R0)