Abstract
By considering agents to be a part of their environment, Orseau and Ring’s space-time embedded intelligence [10] is a better fit to the real world than the traditional agent framework. However, a self-modifying AGI that sees future versions of itself as an ordinary part of the environment may run into problems of self-reference. We show that in one particular model based on formal logic, naive approaches either lead to incorrect reasoning that allows an agent to put off an important task forever (the procrastination paradox), or fail to allow the agent to justify even obviously safe rewrites (the Löbian obstacle). We argue that these problems have relevance beyond our particular formalism, and discuss partial solutions.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Christiano, P., Yudkowsky, E., Herreshoff, M., Barasz, M.: Definability of truth in probabilistic logic (2013), http://intelligence.org/files/DefinabilityOfTruthInProbabilisticLogic-EarlyDraft.pdf
Fallenstein, B.: An infinitely descending sequence of sound theories each proving the next consistent (2013), https://intelligence.org/files/ConsistencyWaterfall.pdf
Fallenstein, B.: Procrastination in probabilistic logic (2014), https://intelligence.org/files/ProbabilisticLogicProcrastinates.pdf
Goertzel, B.: Golem: Toward an agi meta-architecture enabling both goal preservation and radical self-improvement (2010), http://goertzel.org/GOLEM.pdf
Hutter, M.: Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability. Springer, Berlin (2005)
Legg, S., Hutter, M.: A formal measure of machine intelligence. In: Proc. 15th Annual Machine Learning Conference of Belgium and the Netherlands (Benelearn 2006), Ghent, Belgium, pp. 73–80 (2006)
Lob, M.H.: Solution of a problem of Leon Henkin. J. Symb. Log. 20(2), 115–118 (1955)
Muehlhauser, L., Orseau, L.: Laurent Orseau on Artificial General Intelligence (interview) (2013), http://intelligence.org/2013/09/06/laurent-orseau-on-agi/
Neumann, L.J., Morgenstern, O.: Theory of games and economic behavior, vol. 60. Princeton University Press, Princeton (1947)
Orseau, L., Ring, M.: Space-time embedded intelligence. In: Bach, J., Goertzel, B., Iklé, M. (eds.) AGI 2012. LNCS, vol. 7716, pp. 209–218. Springer, Heidelberg (2012)
Robinson, H.: Dualism. In: Zalta, E.N. (ed.) The Stanford Encyclopedia of Philosophy. Winter 2012 edition (2012)
Schmidhuber, J.: Ultimate cognition à la Gödel. Cognitive Computation 1(2), 177–193 (2009)
Yudkowsky, E.: The procrastination paradox (2013), https://intelligence.org/files/ProcrastinationParadox.pdf
Yudkowsky, E., Herreshoff, M.: Tiling agents for self-modifying AI, and the Löbian obstacle (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Fallenstein, B., Soares, N. (2014). Problems of Self-reference in Self-improving Space-Time Embedded Intelligence. In: Goertzel, B., Orseau, L., Snaider, J. (eds) Artificial General Intelligence. AGI 2014. Lecture Notes in Computer Science(), vol 8598. Springer, Cham. https://doi.org/10.1007/978-3-319-09274-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-09274-4_3
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-09273-7
Online ISBN: 978-3-319-09274-4
eBook Packages: Computer ScienceComputer Science (R0)