Skip to main content
SpringerLink
Log in

A Priori Advantages of Meta-Induction and the No Free Lunch Theorem: A Contradiction?

  • Conference paper
  • First Online:
KI 2017: Advances in Artificial Intelligence (KI 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10505))

Abstract

Recently a new account to the problem of induction has been developed [1], based on a priori advantages of regret-weighted meta-induction (RW) in online learning [2]. The claimed a priori advantages seem to contradict the no free lunch (NFL) theorem, which asserts that relative to a state-uniform prior distribution (SUPD) over possible worlds all (non-clairvoyant) prediction methods have the same expected predictive success. In this paper we propose a solution to this problem based on four novel results:

  • RW enjoys free lunches, i.e., its predictive long-run success dominates that of other prediction strategies.

  • Yet the NFL theorem applies to online prediction tasks provided the prior distribution is a SUPD.

  • The SUPD is maximally induction-hostile and assigns a probability of zero to all possible worlds in which RW enjoys free lunches. This dissolves the apparent conflict with the NFL.

  • The a priori advantages of RW can be demonstrated even under the assumption of a SUPD. Further advantages become apparent when a frequency-uniform distribution is considered.

This work was supported by the DFG (Deutsche Forschungsgemeinschaft), SPP 1516. For valuable help we are indebted to Ronald Ortner.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Cf. [6, Sect. 4] and [7, Sect. 3], citing statements from a 1994 e-mail discussion.

  2. 2.

    [8] mentions the weak no free lunch theorem in a small paragraph on p. 1354; for our purpose this NFL theorem is the most important one.

References

  1. Schurz, G.: The meta-inductivist’s winning strategy in the prediction game: a new approach to Hume’s problem. Philos. Sci. 75, 278–305 (2008)

    Article  MathSciNet  Google Scholar 

  2. Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)

    Book  MATH  Google Scholar 

  3. Wolpert, D., Macready, W.: No free lunch theorems for search. Technical report SFI-TR-95-02-010, Santa Fe Institute (1995)

    Google Scholar 

  4. Wolpert, D.: On the connection between in-sample testing and generalization error. Complex Syst. 6, 47–94 (1992)

    MathSciNet  MATH  Google Scholar 

  5. Schaffer, C.: A conservation law for generalization performance. In: Machine Learning (Proceedings of ICML 1994), pp. 259–265. Morgan Kaufmann, Burlington (1994)

    Google Scholar 

  6. Rao, R., Gordon, D., Spears, W.: For every generalization action, is there really an equal and opposite reaction? In: Machine Learning (Proceedings of ICML 1995), pp. 471–479. Morgan Kaufmann, Burlington (1994)

    Google Scholar 

  7. Giraud-Carrier, C., Provost, F.: Toward a justification of meta-learning: is the no free lunch theorem a show-stopper? In: Proceedings of the ICML-2005 Workshop on Meta-learning, pp. 12–19 (2006)

    Google Scholar 

  8. Wolpert, D.: The lack of a priori distinctions between learning algorithms. Neural Comput. 8, 1341–1390 (1996)

    Article  Google Scholar 

  9. Schurz, G., Thorn, P.: The revenge of ecological rationality: strategy-selection by meta-induction. Minds Mach. 26(1), 31–59 (2016)

    Article  Google Scholar 

  10. Schurz, G.: Meta-induction is an optimal prediction strategy. In: Proceedings of the 18th Annual Belgian-Dutch Conference on Machine Learning, pp. 66–74. University of Tilburg (2009)

    Google Scholar 

  11. Gigerenzer, G., Todd, P.M., the ABC Research Group. (eds.): Simple Heuristics That Make Us Smart. Oxford University Press, Dordrecht (1999)

    Google Scholar 

  12. Schurz, G.: The Optimality of Meta-induction. A New Approach to Hume’s Problem. Book manuscript (2017)

    Google Scholar 

  13. Carnap, R.: Logical Foundations of Probability, 2nd edn. University of Chicago Press, Chicago (1962)

    MATH  Google Scholar 

  14. Howson, C., Urbach, P.: Scientific Reasoning: The Bayesian Approach, 2nd edn. Open Court, Chicago (1996)

    Google Scholar 

  15. Solomonoff, R.: A formal theory of inductive inference. Inf. Control 7(1–22), 224–254 (1964)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DCLPS, Heinrich Heine University Düesseldorf, Düesseldorf, Germany

    Gerhard Schurz & Paul Thorn

Authors
  1. Gerhard Schurz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul Thorn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gerhard Schurz .

Editor information

Editors and Affiliations

  1. Fakultät für Informatik, Technische Universität Dortmund, Dortmund, Germany

    Gabriele Kern-Isberner

  2. FB Informatik, TU Darmstadt, Darmstadt, Hessen, Germany

    Johannes Fürnkranz

  3. FB Informatik, Universität Koblenz, Koblenz, Rheinland-Pfalz, Germany

    Matthias Thimm

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Schurz, G., Thorn, P. (2017). A Priori Advantages of Meta-Induction and the No Free Lunch Theorem: A Contradiction?. In: Kern-Isberner, G., Fürnkranz, J., Thimm, M. (eds) KI 2017: Advances in Artificial Intelligence. KI 2017. Lecture Notes in Computer Science(), vol 10505. Springer, Cham. https://doi.org/10.1007/978-3-319-67190-1_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67190-1_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67189-5

  • Online ISBN: 978-3-319-67190-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation