Skip to main content
SpringerLink
Log in
Book cover

50 Years of Artificial Intelligence pp 232–242Cite as

Tests of Machine Intelligence

  • Chapter

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4850))

Abstract

Although the definition and measurement of intelligence is clearly of fundamental importance to the field of artificial intelligence, no general survey of definitions and tests of machine intelligence exists. Indeed few researchers are even aware of alternatives to the Turing test and its many derivatives. In this paper we fill this gap by providing a short survey of the many tests of machine intelligence that have been proposed.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alvarado, N., Adams, S., Burbeck, S., Latta, C.: Beyond the Turing test: Performance metrics for evaluating a computer simulation of the human mind. In: Performance Metrics for Intelligent Systems Workshop, Gaithersburg, MD, USA, North-Holland, Amsterdam (2002)

    Google Scholar 

  2. Bell, T.C., Cleary, J.G., Witten, I.H.: Text compression. Prentice-Hall, Englewood Cliffs (1990)

    Google Scholar 

  3. Block, N.: Psychologism and behaviorism. Philosophical Review 90, 5–43 (1981)

    Article  Google Scholar 

  4. Bringsjord, S., Schimanski, B.: What is artificial intelligence? Psychometric AI as an answer. In: Eighteenth International Joint Conference on Artificial Intelligence, vol. 18, pp. 887–893 (2003)

    Google Scholar 

  5. Chaitin, G.J.: Gödel’s theorem and information. International Journal of Theoretical Physics 22, 941–954 (1982)

    Article  MathSciNet  Google Scholar 

  6. Dowe, D.L., Hajek, A.R.: A non-behavioural, computational extension to the Turing test. In: ICCIMA 1998. International Conference on Computational Intelligence & Multimedia Applications, pp. 101–106. Gippsland, Australia (1998)

    Google Scholar 

  7. Eisner, J.: Cognitive science and the search for intelligence. Invited paper presented to the Socratic Society, University of Cape Town (1991)

    Google Scholar 

  8. Ford, K.M., Hayes, P.J.: On computational wings: Rethinking the goals of artificial intelligence. Scientific American, Special edn. (4) (1998)

    Google Scholar 

  9. French, R.M.: Subcognition and the limits of the Turing test. Mind 99, 53–65 (1990)

    Article  MathSciNet  Google Scholar 

  10. Gottfredson, L.S.: Why g matters: The complexity of everyday life. Intelligence 24(1), 79–132 (1997)

    Article  Google Scholar 

  11. Harnad, S.: Minds, machines and Searle. Journal of Theoretical and Experimental Artificial Intelligence 1, 5–25 (1989)

    Article  Google Scholar 

  12. Hernández-Orallo, J.: Beyond the Turing test. Journal of Logic, Language and Information 9(4), 447–466 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  13. Hernández-Orallo, J.: On the computational measurement of intelligence factors. In: Performance Metrics for Intelligent Systems Workshop, Gaithersburg, MD, USA, pp. 1–8 (2000)

    Google Scholar 

  14. Hernández-Orallo, J., Minaya-Collado, N.: A formal definition of intelligence based on an intensional variant of Kolmogorov complexity. In: EIS 1998. Proceedings of the International Symposium of Engineering of Intelligent Systems, pp. 146–163. ICSC Press (1998)

    Google Scholar 

  15. Hutchens, J.L.: How to pass the Turing test by cheating (1996), www.cs.umbc.edu/471/current/papers/hutchens.pdf

  16. Hutter, M.: Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability, p. 300 Springer, Berlin (2005), http://www.hutter1.net/ai/uaibook.htm

  17. Hutter, M.: The Human knowledge compression prize (2006), http://prize.hutter1.net

  18. Legg, S., Hutter, M.: A collection of definitions of intelligence. In: Goertzel, B. (ed.) Proc. 1st Annual artificial general intelligence workshop (to appear), Online version www.idsia.ch/ shane/intelligence.html

    Google Scholar 

  19. Legg, S., Hutter, M.: A formal measure of machine intelligence. In: Benelearn 2006. Annual Machine Learning Conference of Belgium and The Netherlands, Ghent (2006)

    Google Scholar 

  20. Levin, L.A.: Universal sequential search problems. Problems of Information Transmission 9, 265–266 (1973)

    Google Scholar 

  21. Li, M., Vitányi, P.M.B.: An introduction to Kolmogorov complexity and its applications, 2nd edn. Springer, Heidelberg (1997)

    MATH  Google Scholar 

  22. Loebner, H.G.: The Loebner prize — The first Turing test (1990), http://www.loebner.net/Prizef/loebner-prize.html

  23. Mahoney, M.V.: Text compression as a test for artificial intelligence. In: AAAI/IAAI (1999)

    Google Scholar 

  24. Masum, H., Christensen, S., Oppacher, F.: The Turing ratio: Metrics for open-ended tasks. In: GECCO 2002. Proceedings of the Genetic and Evolutionary Computation Conference, pp. 973–980. Morgan Kaufmann Publishers, New York (2002)

    Google Scholar 

  25. Sanghi, P., Dowe, D.L.: A computer program capable of passing I.Q. tests. In: Sloot, P.M.A., Abramson, D., Bogdanov, A.V., Gorbachev, Y.E., Dongarra, J.J., Zomaya, A.Y. (eds.) ICCS 2003. LNCS, vol. 2657, pp. 570–575. Springer, Heidelberg (2003)

    Google Scholar 

  26. Saygin, A., Cicekli, I., Akman, V.: Turing test: 50 years later. Minds and Machines 10 (2000)

    Google Scholar 

  27. Schweizer, P.: The truly total Turing test. Minds and Machines 8, 263–272 (1998)

    Article  Google Scholar 

  28. Searle, J.: Minds, brains, and programs. Behavioral & Brain Sciences 3, 417–458 (1980)

    Article  Google Scholar 

  29. Shieber, S.: Lessons from a restricted Turing test. CACM: Communications of the ACM 37 (1994)

    Google Scholar 

  30. Smith, W.D.: Mathematical definition of “intelligence” (and consequences) (2006), http://math.temple.edu/~wds/homepage/works.html

  31. Treister-Goren, A., Dunietz, J., Hutchens, J.L.: The developmental approach to evaluating artificial intelligence – a proposal. In: Performance Metrics for Intelligence Systems (2000)

    Google Scholar 

  32. Treister-Goren, A., Hutchens, J.L.: Creating AI: A unique interplay between the development of learning algorithms and their education. In: Proceeding of the First International Workshop on Epigenetic Robotics (2001)

    Google Scholar 

  33. Turing, A.M.: Computing machinery and intelligence. Mind (October 1950)

    Google Scholar 

  34. Voss, P.: Essentials of general intelligence: The direct path to AGI. In: Goertzel, B., Pennachin, C. (eds.) Artificial General Intelligence, Springer, Heidelberg (2005)

    Google Scholar 

  35. Watt, S.: Naive psychology and the inverted Turing test. Psycoloquy 7(14) (1996)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IDSIA, Galleria 2, Manno-Lugano CH-6928, Switzerland

    Shane Legg

  2. RSISE/ANU/NICTA, Canberra, ACT, 0200, Australia

    Marcus Hutter

Authors
  1. Shane Legg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcus Hutter
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Max Lungarella Fumiya Iida Josh Bongard Rolf Pfeifer

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Legg, S., Hutter, M. (2007). Tests of Machine Intelligence. In: Lungarella, M., Iida, F., Bongard, J., Pfeifer, R. (eds) 50 Years of Artificial Intelligence. Lecture Notes in Computer Science(), vol 4850. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77296-5_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-77296-5_22

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-77295-8

  • Online ISBN: 978-3-540-77296-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation