Abstract
Self-improving software has been a goal of computer scientists since the founding of the field of Artificial Intelligence. In this work we analyze limits on computation which might restrict recursive self-improvement. We also introduce Convergence Theory which aims to predict general behavior of RSI systems.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Yampolskiy, R.V., Construction of an NP Problem with an Exponential Lower Bound (2011). Arxiv preprint arXiv:1111.0305
Yonck, R.: Toward a Standard Metric of Machine Intelligence. World Future Review 4(2), 61–70 (2012)
Bremermann, H.J.: Quantum noise and information. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability (1967)
Bekenstein, J.D.: Information in the holographic universe. Scientific American 289(2), 58–65 (2003)
Lloyd, S.: Ultimate Physical Limits to Computation. Nature 406, 1047–1054 (2000)
Sandberg, A.: The physics of information processing superobjects: daily life among the Jupiter brains. Journal of Evolution and Technology 5(1), 1–34 (1999)
Aaronson, S.: Guest column: NP-complete problems and physical reality. ACM Sigact News 36(1), 30–52 (2005)
Shannon, C.E.: A Mathematical Theory of Communication. Bell Systems Technical Journal 27(3), 379–423 (1948)
Krauss, L.M., Starkman, G.D.: Universal limits on computation (2004). arXiv preprint astro-ph/0404510
Fox, D.: The limits of intelligence. Scientific American 305(1), 36–43 (2011)
Einstein, A.: Does the inertia of a body depend upon its energy-content? Annalen der Physik 18, 639–641 (1905)
Wheeler, J.A.: Information, Physics, Quantum: The Search for Links. Univ. of Texas (1990)
Schaeffer, J., et al.: Checkers is Solved. Science 317(5844), 1518–1522 (2007)
Mahoney, M.: Is there a model for RSI?. In: SL4, June 20, 2008. http://www.sl4.org/archive/0806/19028.html
Turing, A.: On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London Mathematical Society 2(42), 230–265 (1936)
Wiedermann, J.: A Computability Argument Against Superintelligence. Cognitive Computation 4(3), 236–245 (2012)
Wiedermann, J.: Is There Something Beyond AI? Frequently Emerging, but Seldom Answered Questions about Artificial Super-Intelligence, p. 76. Artificial Dreams, Beyond AI
Mahoney, M.: A Model for Recursively Self Improving Programs (2010). http://mattmahoney.net/rsi.pdf
Yudkowsky, E., Intelligence Explosion Microeconomics. In: MIRI Technical Report. www.intelligence.org/files/IEM.pdf
Rice, H.G.: Classes of recursively enumerable sets and their decision problems. Transactions of the American Mathematical Society 74(2), 358–366 (1953)
Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation 1(1), 67–82 (1997)
Melkikh, A.V.: The No Free Lunch Theorem and hypothesis of instinctive animal behavior. Artificial Intelligence Research 3(4), p43 (2014)
de Garis, H.: The 21st. Century Artilect: Moral Dilemmas Concerning the Ultra Intelligent Machine. Revue Internationale de Philosophie 44(172), 131–138 (1990)
Yudkowsky, E., Herreshoff, M.: Tiling agents for self-modifying AI, and the Löbian obstacle. In: MIRI Technical Report (2013)
Fallenstein, B., Soares, N.: Problems of self-reference in self-improving space-time embedded intelligence. In: MIRI Technical Report (2014)
Yudkowsky, E.: The Procrastination Paradox (Brief technical note). In: MIRI Technical Report (2014). https://intelligence.org/files/ProcrastinationParadox.pdf
Bolander, T.: Logical theories for agent introspection. Comp. Science 70(5), 2002 (2003)
Orseau, L.: Ring, M.: Self-modification and mortality in artificial agents. In: 4th international conference on Artificial general intelligence, pp. 1–10. Mount. View, CA. (2011)
Yampolskiy, R.V.: Utility Function Security in Artificially Intelligent Agents. Journal of Experimental and Theoretical Artificial Intelligence (JETAI), 1–17 (2014)
Chalmers, D.: The Singularity: A Philosophical Analysis. Journal of Consciousness Studies 17, 7–65 (2010)
Yampolskiy, R.V.: Artificial intelligence safety engineering: Why machine ethics is a wrong approach. In: Philosophy and Theory of Artificial Intelligence, pp. 389–396, Springer (2013)
Wolfram, S.: A New Kind of Science. Wolfram Media, Inc., May 14, 2002
Yampolskiy, R.V.: Computing Partial Solutions to Difficult AI Problems. In: Midwest Artificial Intelligence and Cognitive Science Conference, p. 90 (2012)
Böckenhauer, H.-J., Hromkovič, J., Mömke, T., Widmayer, P.: On the hardness of reoptimization. In: Geffert, V., Karhumäki, J., Bertoni, A., Preneel, B., Návrat, P., Bieliková, M. (eds.) SOFSEM 2008. LNCS, vol. 4910, pp. 50–65. Springer, Heidelberg (2008)
Ausiello, G., Escoffier, B., Monnot, J., Paschos, V.T.: Reoptimization of minimum and maximum traveling salesman’s tours. In: Arge, L., Freivalds, R. (eds.) SWAT 2006. LNCS, vol. 4059, pp. 196–207. Springer, Heidelberg (2006)
Archetti, C., Bertazzi, L., Speranza, M.G.: Reoptimizing the traveling salesman problem. Networks 42(3), 154–159 (2003)
Ausiello, G., Bonifaci, V., Escoffier, B.: Complexity and approximation in reoptimization. Imperial College Press/World Scientific (2011)
Loosemore, R., Goertzel, B.: Why an intelligence explosion is probable. In: Singularity Hypotheses, pp. 83–98. Springer (2012)
Shahaf, D., Amir, E.: Towards a theory of AI completeness. In: 8th International Symposium on Logical Formalizations of Commonsense Reasoning. California, March 26–28, 2007
Yampolskiy, R.V.: Turing test as a defining feature of AI-completeness. In: Yang, X.-S. (ed.) Artificial Intelligence, Evolutionary Computing and Metaheuristics. SCI, vol. 427, pp. 3–17. Springer, Heidelberg (2013)
Yampolskiy, R.V.: AI-complete, AI-hard, or AI-easy–classification of problems in AI. In: The 23rd Midwest Artificial Intelligence and Cognitive Science Conference, OH, USA (2012)
Yudkowsky, E.S.: General Intelligence and Seed AI (2001). http://singinst.org/ourresearch/publications/GISAI/
Yampolskiy, R.V.: Efficiency Theory: a Unifying Theory for Information, Computation and Intelligence. J. of Discrete Math. Sciences & Cryptography 16(4–5), 259–277 (2013)
Yampolskiy, R.V.: AI-Complete CAPTCHAs as Zero Knowledge Proofs of Access to an Artificially Intelligent System. ISRN Artificial Intelligence 271878 (2011)
Turing, A.M.: On Computable Numbers, with an Application to the Entscheidungsproblem. Proceedings of the London Mathematical Society 42, 230–265 (1936)
Bostrom, N.: Superintelligence: Paths, dangers, strategies. Oxford University Press (2014)
Hall, J.S.: Engineering utopia. Frontiers in AI and Applications 171, 460 (2008)
Hutter, M.: Universal algorithmic intelligence: A mathematical top→ down approach. In: Artificial general intelligence, pp. 227–290. Springer (2007)
Kolmogorov, A.N.: Three Approaches to the Quantitative Definition of Information. Problems Inform. Transmission 1(1), 1–7 (1965)
Yampolskiy, R.V.: The Universe of Minds (2014). arXiv:1410.0369
Yudkowsky, E.: Levels of organization in general intelligence. In: Artificial general intelligence, pp. 389–501. Springer (2007)
Bostrom, N.: What is a Singleton? Linguistic and Philosophical Invest. 5(2), 48–54 (2006)
Yudkowsky, E.: Timeless decision theory. The Singularity Institute, San Francisco (2010)
LessWrong: Acausal Trade, September 29, 2014. http://wiki.lesswrong.com/wiki/Acausal_trade
Yudkowsky, E.S.: Coherent Extrapolated Volition. Singularity Institute for Artificial Intelligence, May 2004. http://singinst.org/upload/CEV.html
Yudkowsky, E.: Recursive Self-Improvement. In: Less Wrong, December 1, 2008. http://lesswrong.com/lw/we/recursive_selfimprovement/, September 29, 2014
Hutter, M.: Can Intelligence Explode? J. of Consciousness Studies 19(1–2), 1–2 (2012)
Yampolskiy, R.V.: Analysis of types of self-improving software. In: The Eighth Conference on Artificial General Intelligence, Berlin, Germany, July 22–25, 2015
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Yampolskiy, R.V. (2015). On the Limits of Recursively Self-Improving AGI. In: Bieger, J., Goertzel, B., Potapov, A. (eds) Artificial General Intelligence. AGI 2015. Lecture Notes in Computer Science(), vol 9205. Springer, Cham. https://doi.org/10.1007/978-3-319-21365-1_40
Download citation
DOI: https://doi.org/10.1007/978-3-319-21365-1_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-21364-4
Online ISBN: 978-3-319-21365-1
eBook Packages: Computer ScienceComputer Science (R0)