Abstract
Given the likely large impact of artificial general intelligence, a formal theory of intelligence is desirable. To further this research program, we present a representation theorem governing the integration of causal models with decision theory. This theorem puts formal bounds on the applicability of the submodel hypothesis, a normative theory of decision counterfactuals that has previously been argued on a priori and practical grounds, as well as by comparison to theories of counterfactual cognition in humans. We are able to prove four conditions under which the submodel hypothesis holds, forcing any preference between acts to be consistent with some utility function over causal submodels.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arel, I.: Reward Driven Learning and the Risk of an Adversarial Artificial General Intelligence. Talk at “The Future of AGI Workshop Part 1 - Ethics of Advanced AGI,” The Fourth Conference on Artificial General Intelligence (2011)
Bostrom, N.: The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents. Minds and Machines 22, 71–85 (2012)
Chalmers, D.: The Singularity: A Philosophical Analysis. Journal of Consciousness Studies 17, 7–65 (2010)
Chalmers, D.: The Singularity: a Reply. Journal of Consciousness Studies 19 (2012)
Drescher, G.: Good and real: Demystifying paradoxes from physics to ethics. Bradford Books, MIT Press, Cambridge, MA (2006)
Galles, D., Pearl, J.: An Axiomatic Characterization of Counterfactuals. Foundations of Science III, 151–182 (1998)
Goertzel, B.: Should Humanity Build a Global AI Nanny to Delay the Singularity Until It’s Better Understood? Journal of Consciousness Studies 19, 96–111 (2012)
Goertzel, B.: Nine Ways to Bias Open-Source AGI Toward Friendliness. Journal of Evolution and Technology 22, 116–131 (2012)
Good, I.J.: Speculations Concerning the First Ultraintelligent Machine. In: Alt, F.L., Rubinoff, M. (eds.) Advances in Computers, vol. 6, pp. 31–88 (1965)
Hutter, M.: Can Intelligence Explode? Journal of Consciousness Studies 19, 143–166 (2012)
Hutter, M.: Universal algorithmic intelligence: A mathematical top-down approach. In: Artificial General Intelligence, pp. 227–290. Springer, Berlin (2007)
Legg, S.: Is there an Elegant Universal Theory of Prediction? IDSIA Technical Report No. IDSIA-12-06 (2006)
Legg, S.: Machine Super Intelligence. PhD dissertation, University of Lugano (2008)
Muehlhauser, L., Salamon, A.: Intelligence Explosion: Evidence and Import. In: The Singularity Hypothesis: A Scientific and Philosophical Assessment. Springer, Berlin (2012)
Pearl, J.: Causality: Models, Reasoning, and Inference. Cambridge University Press (2000)
Russell, S., Norvig, P.: AI: A Modern Approach, 3rd edn. Prentice-Hall, Englewood Cliffs (1995)
Schmidhuber, J.: Philosophers & Futurists, Catch Up! Journal of Consciousness Studies 19, 173–182 (2012)
Schmidhuber, J.: Gödel machines: Fully Self-Referential Optimal Universal Self-Improvers. In: Artificial General Intelligence, pp. 119–226 (2006)
Solomonoff, R.: A Formal Theory of Inductive Inference, Part I. Information and Control 7(1), 1–22 (1964)
Solomonoff, R.: A Formal Theory of Inductive Inference, Part II. Information and Control 7(2), 224–254 (1964)
Voorneveld, M.: Mathematical Foundations of Microeconomic Theory: Preference, Utility, Choice (2010), https://studentweb.hhs.se/CourseWeb/CourseWeb/Public/PhD501/1001/micro1.pdf
Yudkowsky, E.: Artificial intelligence as a positive and negative factor in global risk. In: Global Catastrophic Risks. Oxford University Press, Oxford (2008)
Yudkowsky, E.: Complex Value Systems are Required to Realize Valuable Futures. In: The Proceedings of the Fourth Conference on Artificial General Intelligence (2011)
Yudkowsky, E., et al.: Reducing Long-Term Catastrophic Risks from Artificial Intelligence. The Singularity Institute, San Francisco (2010)
Yudkowsky, E.: Timeless decision theory. The Singularity Institute, San Francisco (2010)
Yudkowsky, E.: Ingredients of Timeless Decision Theory (2009), http://lesswrong.com/lw/15z/ingredients_of_timeless_decision_theory/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dewey, D. (2012). A Representation Theorem for Decisions about Causal Models. In: Bach, J., Goertzel, B., Iklé, M. (eds) Artificial General Intelligence. AGI 2012. Lecture Notes in Computer Science(), vol 7716. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35506-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-35506-6_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35505-9
Online ISBN: 978-3-642-35506-6
eBook Packages: Computer ScienceComputer Science (R0)