Abstract
Perfectly rational decision-makers maximize expected utility, but crucially ignore the resource costs incurred when determining optimal actions. Here we employ an axiomatic framework for bounded rational decision-making based on a thermodynamic interpretation of resource costs as information costs. This leads to a variational “free utility” principle akin to thermodynamical free energy that trades off utility and information costs. We show that bounded optimal control solutions can be derived from this variational principle, which leads in general to stochastic policies. Furthermore, we show that risk-sensitive and robust (minimax) control schemes fall out naturally from this framework if the environment is considered as a bounded rational and perfectly rational opponent, respectively. When resource costs are ignored, the maximum expected utility principle is recovered.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Basar, T., Bernhard, P.: H-Infinity Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach. Birkhäuser, Boston (1995)
Braun, D.A., Ortega, P.A.: A minimum relative entropy principle for adaptive control in linear quadratic regulators. In: The 7th Conference on Informatics in Control, Automation and Robotics, vol. 3, pp. 103–108 (2010)
Braun, D.A., Ortega, P.A., Theodorou, E., Schaal, S.: Path integral control and bounded rationality. In: IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (2011)
Callen, H.B.: Thermodynamics and an Introduction to Themostatistics, 2nd edn. John Wiley & Sons, Chichester (1985)
Feynman, R.P.: The Feynman Lectures on Computation. Addison-Wesley, Reading (1996)
Haussler, D., Opper, M.: Mutual information, metric entropy and cumulative relative entropy risk. The Annals of Statistics 25, 2451–2492 (1997)
Kappen, B.: A linear theory for control of non-linear stochastic systems. Physical Review Letters 95, 200–201 (2005)
Keller, G.: Equilibrium States in Ergodic Theory. London Mathematical Society Student Texts. Cambridge University Press, Cambridge (1998)
Ortega, P.A., Braun, D.A.: A minimum relative entropy principle for learning and acting. Journal of Artificial Intelligence Research 38, 475–511 (2010)
Ortega, P.A., Braun, D.A.: A bayesian rule for adaptive control based on causal interventions. In: The Third Conference on Artificial General Intelligence, pp. 121–126. Atlantis Press, Paris (2010)
Ortega, P.A., Braun, D.A.: A conversion between utility and information. In: The Third Conference on Artificial General Intelligence, pp. 115–120. Atlantis Press, Paris (2010)
Simon, H.: Models of Bounded Rationality. MIT Press, Cambridge (1982)
Still, S.: An information-theoretic approach to interactive learning. Europhysics Letters 85, 28005 (2009)
Todorov, E.: Linearly solvable markov decision problems. In: Advances in Neural Information Processing Systems, vol. 19, pp. 1369–1376 (2006)
Todorov, E.: Efficient computation of optimal actions. Proceedings of the National Academy of Sciences USA 106, 11478–11483 (2009)
Tribus, M., McIrvine, E.C.: Energy and information. Scientific American 225, 179–188 (1971)
Vitanyi, P.M.B.: Time, space, and energy in reversible computing. In: Proceedings of the 2nd ACM Conference on Computing Frontiers, pp. 435–444 (2005)
von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior. Princeton University Press, Princeton (1944)
Whittle, P.: Risk-sensitive optimal control. John Wiley and Sons, Chichester (1990)
Wolpert, D.H.: Information theory - the bridge connecting bounded rational game theory and statistical physics. In: Braha, D., Bar-Yam, Y. (eds.) Complex Engineering Systems. Perseus Books, Cambridge (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ortega, D.A., Braun, P.A. (2011). Information, Utility and Bounded Rationality. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds) Artificial General Intelligence. AGI 2011. Lecture Notes in Computer Science(), vol 6830. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22887-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-642-22887-2_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22886-5
Online ISBN: 978-3-642-22887-2
eBook Packages: Computer ScienceComputer Science (R0)