Skip to main content
SpringerLink
Log in
Book cover

International Conference on Artificial General Intelligence

AGI 2013: Artificial General Intelligence pp 150–159Cite as

Learning Agents with Evolving Hypothesis Classes

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7999))

Abstract

It has recently been shown that a Bayesian agent with a universal hypothesis class resolves most induction problems discussed in the philosophy of science. These ideal agents are, however, neither practical nor a good model for how real science works. We here introduce a framework for learning based on implicit beliefs over all possible hypotheses and limited sets of explicit theories sampled from an implicit distribution represented only by the process by which it generates new hypotheses. We address the questions of how to act based on a limited set of theories as well as what an ideal sampling process should be like. Finally, we discuss topics in philosophy of science and cognitive science from the perspective of this framework.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   49.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Asmuth, J., Li, L., Littman, M.L., Nouri, A., Wingate, D.: A bayesian sampling approach to exploration in reinforcement learning. In: Uncertainty in Artificial Intelligence (UAI), pp. 19–26 (2009)

    Google Scholar 

  2. Anderson, J.R.: Is human cognition adaptive? Behavioral & Brain Sciences 14(3), 471–517 (1991)

    Article  Google Scholar 

  3. Berg, B.A.: Markov Chain Monte Carlo Simulations And Their Statistical Analysis: With Web-based Fortran Code. World Scientific Publishing Company (2004)

    Google Scholar 

  4. Box, G.E.P., Tiao, G.C.: Bayesian Inference in Statistical Analysis (Wiley Classics Library). Wiley-Interscience (1992)

    Google Scholar 

  5. Casella, G., George, E.I.: Explaining the gibbs sampler. The American Statistician 46(3), 167–174 (1992)

    MathSciNet  Google Scholar 

  6. Demski, A.: Logical prior probability. In: Bach, J., Goertzel, B., Iklé, M. (eds.) AGI 2012. LNCS, vol. 7716, pp. 50–59. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  7. Edwards, W.: Conservatism in human information processing. In: Formal Representation of Human Judgment, pp. 17–52 (1968)

    Google Scholar 

  8. Griffiths, T.L., Kemp, C., Tenenbaum, J.B.: Bayesian Models of Cognition. Cambridge University Press (2008)

    Google Scholar 

  9. Good, I.J.: A Bayesian approach in the philosophy of inference. British Journal for the Philosophy of Science, 161–166 (1984)

    Google Scholar 

  10. Gaifman, H., Snir, M.: Probabilities over rich languages, testing and randomness. J. Symb. Log. 47(3), 495–548 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  11. Griffiths, T.L., Tenenbaum, J.B.: Theory-based causal induction. Psychological Review 116(4), 661–716 (2009)

    Article  Google Scholar 

  12. Hanson, N.R.: Patterns of Discovery. Cambridge University Press, Cambridge (1958)

    Google Scholar 

  13. Hutter, M., Lloyd, J.W., Ng, K.S., Uther, W.T.B.: Probabilities on sentences in an expressive logic. Journal of Applied Probability (2013)

    Google Scholar 

  14. Holland, J.H.: Adaptation in Natural and Artificial Systems. University of Michigan Press (1975)

    Google Scholar 

  15. Howson, C., Urbach, P.: Scientific Reasoning: The Bayesian Approach, 3rd edn., Open Court (2005)

    Google Scholar 

  16. Hutter, M.: Universal Articial Intelligence: Sequential Decisions based on Algorithmic Probability. Springer, Berlin (2005)

    Google Scholar 

  17. Hutter, M.: On universal prediction and bayesian confirmation. Theoretical Computer Science 384, 33–48 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  18. Hutter, M.: Feature reinforcement learning: Part I. Unstructured MDPs. Journal of General Artificial Intelligence (2009)

    Google Scholar 

  19. Hutter, M.: The subjective computable universe. In: A Computable Universe: Understanding and Exploring Nature as Computation, pp. 399–416. World Scientific (2012)

    Google Scholar 

  20. Jaynes, E.T.: Probability Theory: The Logic of Science. Cambridge University Press (2003)

    Google Scholar 

  21. Kahneman, D., Tversky, A.: Prospect theory: An analysis of decision under risk. Econometrica 47, 263–291 (1979)

    Article  MATH  Google Scholar 

  22. Kemp, C., Tenenbaum, J.B., Niyogi, S., Griffiths, T.L.: A probabilistic model of theory formation. Cognition 114(2) (2010)

    Google Scholar 

  23. Kuhn, T.S.: The structure of scientific revolutions. University of Chicago Press (1970)

    Google Scholar 

  24. Lattimore, T., Hutter, M.: Asymptotically optimal agents. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 368–382. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  25. Li, M., Vitany, P.: An Introduction to Kolmogov Complexity and Its Applications. Springer (1993)

    Google Scholar 

  26. Muggleton, S.: Inductive logic programming. New Generation Computing 8(4), 295–318 (1991)

    Article  MATH  Google Scholar 

  27. Nestler, S., Blank, H., von Collani, G.: Hindsight bias doesn’t always come easy: Causal models, cognitive effort, and creeping determinism. Journal of Experimental Psychology: Learning 34(5) (2008)

    Google Scholar 

  28. Nyarko, Y.: The savage-bayesian foundations of economic dynamics. Working Papers 93-35, C.V. Starr Center for Applied Economics, New York University (1993)

    Google Scholar 

  29. Orseau, L., Lattimore, T.: Univeral knowledge-seeking agents for stochastic environments (submitted, 2013)

    Google Scholar 

  30. Osborne, M.J., Rubinstein, A.: A Course in Game Theory. MIT Press Books. The MIT Press (1994)

    Google Scholar 

  31. Orseau, L.: Universal knowledge-seeking agents. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 353–367. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  32. Rechenberg, I.: Evolutionsstrategie. Frommann-Holzboog-Verlag (1971)

    Google Scholar 

  33. Rathmanner, S., Hutter, M.: A philosophical treatise of universal induction. Entropy 13(6), 1076–1136 (2011)

    Article  MathSciNet  Google Scholar 

  34. Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach, 3rd edn. Prentice Hall, Englewood Cliffs (2010)

    Google Scholar 

  35. Robert, C.P.: The Bayesian choice: a decision-theoretic motivation. Springer, New York (1994)

    Book  MATH  Google Scholar 

  36. Savage, L.: The Foundations of Statistics. Wiley, New York (1954)

    MATH  Google Scholar 

  37. Sunehag, P., Hutter, M.: Axioms for rational reinforcement learning. In: Kivinen, J., Szepesvári, C., Ukkonen, E., Zeugmann, T. (eds.) ALT 2011. LNCS, vol. 6925, pp. 338–352. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  38. Sunehag, P., Hutter, M.: Optimistic agents are asymptotically optimal. In: Thielscher, M., Zhang, D. (eds.) AI 2012. LNCS, vol. 7691, pp. 15–26. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  39. Sunehag, P., Hutter, M.: Rational general reinforcement learning (submitted, 2013)

    Google Scholar 

  40. Solomonoff, R.J.: A formal theory of inductive inference. Part i and ii. Information and Control 7(1,2), 1–22, 224–254 (1964)

    Google Scholar 

  41. Kyle Stanford, P.: Exceeding Our Grasp: Science, History, and the Problem of Unconceived Alternatives. Oxford University Press (2006)

    Google Scholar 

  42. Strens, M.J.A.: A bayesian framework for reinforcement learning. In: Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000), pp. 943–950 (2000)

    Google Scholar 

  43. Strevens, M.: The bayesian approach to the philosophy of science. In: Encyclopedia of Philosophy, 2nd edn. (2006)

    Google Scholar 

  44. Tenenbaum, J.B., Kemp, C., Griffiths, T.L., Goodman, N.D.: How to Grow a Mind: Statistics, Structure, and Abstraction. Science 331(6022), 1279–1285 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  45. Weinstein, N.D.: Unrealistic optimism about future life events. Journal of Personality and Social Psychology 39(5), 806–820 (1980)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research School of Computer Science, Australian National University, Canberra, Australia

    Peter Sunehag & Marcus Hutter

Authors
  1. Peter Sunehag
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcus Hutter
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Osnabrück, Albrechtstr. 28, Germany

    Kai-Uwe Kühnberger

  2. Technische Universität Dresden, 01069, Dresden, Germany

    Sebastian Rudolph

  3. Department of Computer and Information Sciences, Temple University, Philadelphia, USA

    Pei Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Sunehag, P., Hutter, M. (2013). Learning Agents with Evolving Hypothesis Classes. In: Kühnberger, KU., Rudolph, S., Wang, P. (eds) Artificial General Intelligence. AGI 2013. Lecture Notes in Computer Science(), vol 7999. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39521-5_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-39521-5_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-39520-8

  • Online ISBN: 978-3-642-39521-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation