Skip to main content
SpringerLink
Log in

A frequentist framework of inductive reasoning

  • Published:
Sankhya A Aims and scope Submit manuscript

Abstract

A betting game establishes a sense in which confidence measures, confidence distributions in the form of probability measures, are the only reliable inferential probability distributions. In addition, because confidence measures are Kolmogorov probability distributions, they are as coherent as Bayesian posterior distributions in their avoidance of sure loss under the usual Dutch-book betting game.

Although a confidence measure can be computed without any prior, previous knowledge can be incorporated into confidence-based reasoning by combining the confidence measure from the observed data with one or more independent confidence measures representing previous agent opinion. The representation of subjective knowledge in terms of confidence measures rather than more general priors preserves approximate frequentist validity and thus reliability in the first game.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Armendt, B. (1992). Dutch strategies for diachronic rules: when believers see the sure loss coming. In PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association 1992, pp. 217–229.

  • Barnard, G.A. (1987). R. A. Fisher: a true Bayesian? International Statistical Review, 55, 183–189.

    Article  MathSciNet  MATH  Google Scholar 

  • Barndorff-nielsen, O.E. and Cox, D.R. (1994). Inference and Asymptotics. CRC Press, London.

    MATH  Google Scholar 

  • Berger, J.O., Bernardo, J.M. and Sun, D. (2009). The formal definition of reference priors. Ann. Statist., 37, 905–938.

    Article  MathSciNet  MATH  Google Scholar 

  • Berger, J.O. (2004). The case for objective Bayesian analysis. Bayesian Anal., 1, 1–17.

    Google Scholar 

  • Bernardo, J.M. (1979). Reference posterior distributions for Bayesian inference. J. R. Stat. Soc. Ser. B, 41, 113–147.

    MathSciNet  MATH  Google Scholar 

  • Bernardo, J.M. (1997). Noninformative priors do not exist: a discussion. J. Statist. Plann. Inference, 65, 159–189.

    Article  MathSciNet  Google Scholar 

  • Bickel, D.R. (2006). Incorporating expert knowledge into frequentist results by combining subjective prior and objective posterior distributions: a generalization of confidence distribution combination. Technical Report, Pioneer Hi-Bred International, arXiv:math.ST/0602377v2.

  • Bickel, D.R. (2011). Estimating the null distribution to adjust observed confidence levels for genome-scale screening. Biometrics, 67, 363–370.

    Article  MathSciNet  MATH  Google Scholar 

  • Bickel, D.R. (2012a). Blending Bayesian and frequentist methods according to the precision of prior information with applications to hypothesis testing. Working Paper, University of Ottawa, deposited in uO Research at http://hdl.handle.net/10393/23124.

  • Bickel, D.R. (2012b). Coherent frequentism: a decision theory based on confidence sets. Comm. Statist. Theory Methods, 41, 1478–1496.

    Article  MathSciNet  MATH  Google Scholar 

  • Bickel, D.R. (2012c). Controlling the degree of caution in statistical inference with the Bayesian and frequentist approaches as opposite extremes. Electron. J. Stat., 6, 686–709.

    Article  Google Scholar 

  • Bickel, D.R. (2012d). Empirical Bayes interval estimates that are conditionally equal to unadjusted confidence intervals or to default prior credibility intervals. Stat. Appl. Genet. Mol. Biol., 11, art. 3.

    Google Scholar 

  • Bickel, D.R. (2012e). A prior-free framework of coherent inference and its derivation of simple shrinkage estimators. Working Paper, University of Ottawa, deposited in uO Research at http://hdl.handle.net/10393/23093.

  • Bickel, D.R. (2012f). The strength of statistical evidence for composite hypotheses: Inference to the best explanation. Statistica Sinica, 22, 1147–1198.

    MathSciNet  MATH  Google Scholar 

  • Brazzale, A.R., Davison, A.C. and Reid, N. (2007). Applied Asymptotics: Case Studies in Small-sample Statistics. Cambridge University Press, Cambridge.

    Book  MATH  Google Scholar 

  • Buehler, R.J. (1977). Conditional confidence statements and confidence estimators: Comment. J. Amer. Statist. Assoc., 72, 813–814.

    MathSciNet  Google Scholar 

  • Carnap, R. (1971). A basic system of inductive logic, part 1. Studies in Inductive Logic and Probability, Vol. 1. University of California Press, Berkeley, pp. 3–165.

    Google Scholar 

  • Chaloner, K. (1996). The elicitation of prior distributions. Bayesian Biostatistics. Marcel Dekker, New York.

    Google Scholar 

  • Clarke, B. (2007). Information optimality and Bayesian modelling. J. Econometrics, 138, 405–429.

    Article  MathSciNet  Google Scholar 

  • Cornfield, J. (1969). The Bayesian outlook and its application. Biometrics, 25, 617–657.

    Article  MathSciNet  Google Scholar 

  • Cox, D.R. (1958). Some problems connected with statistical inference. Annals of Mathematical Statistics, 29, 357–372.

    Article  MathSciNet  MATH  Google Scholar 

  • Craig, P.S., Goldstein, M., Seheult, A.H. and Smith, J.A. (1998). Constructing partial prior specifications for models of complex physical systems. The Statistician, 47, 37–53.

    Google Scholar 

  • Datta, G.S. and Mukerjee, R. (2004). Probability Matching Priors: Higher Order Asymptotics. Springer, New York.

    Book  MATH  Google Scholar 

  • De Finetti, B. (1970). Theory of Probability: a Critical Introductory Treatment, 1st Edition. John Wiley and Sons Ltd, New York.

    Google Scholar 

  • Dempster, A.P. (2008). The Dempster-Shafer calculus for statisticians. Internat. J. Approx. Reason., 48, 365–377.

    Article  MathSciNet  MATH  Google Scholar 

  • Edwards, A.W.F. (1992). Likelihood. Johns Hopkins Press, Baltimore.

    MATH  Google Scholar 

  • Efron, B. (1993). Bayes and likelihood calculations from confidence intervals. Biometrika, 80, 3–26.

    Article  MathSciNet  MATH  Google Scholar 

  • Efron, B. (1998). R. A. Fisher in the 21st century, invited paper presented at the 1996 R. A. Fisher lecture. Statist. Sci., 13, 95–114.

    Article  MathSciNet  MATH  Google Scholar 

  • Efron, B. and Tibshirani, R. (1998). The problem of regions. Ann. Statist. 26, 1687–1718.

    Article  MathSciNet  MATH  Google Scholar 

  • Fisher, R.A. (1960). Scientific thought and the refinement of human reasoning. J. Oper. Res. Soc. Japan, 3, 1–10.

    Google Scholar 

  • Fisher, R.A. (1973). Statistical Methods and Scientific Inference. Hafner Press, New York.

    MATH  Google Scholar 

  • Fraser, D. (1978). Inference and Linear Models. McGraw-Hill, New York.

    Google Scholar 

  • Fraser, D.A.S. (1968). The Structure of Inference. John Wiley, New York.

    MATH  Google Scholar 

  • Fraser, D.A.S. (1977). Confidence, posterior probability, and the buehler example. Ann. Statist., 5, 892–898.

    Article  MathSciNet  MATH  Google Scholar 

  • Fraser, D.A.S. (1991). Statistical inference: likelihood to significance. J. Amer. Statist. Assoc., 86, 258–265.

    Article  MathSciNet  MATH  Google Scholar 

  • Fraser, D.A.S. (2004). Ancillaries and conditional inference. Statist. Sci., 19, 333–351.

    Article  MathSciNet  MATH  Google Scholar 

  • Fraser, D.A.S. (2006). Did Lindley get the argument the wrong way around? Technical Report, Department of Statistics, University of Toronto.

  • Fraser, D.A.S. and Reid, N. (2002). Strong matching of frequentist and Bayesian parametric inference. J. Statist. Plann. Inference, 103, 263–285.

    Article  MathSciNet  MATH  Google Scholar 

  • Freedman, D.A. and Purves, R.A. (1969). Bayes’ method for bookies. Annals of Mathematical Statistics, 40, 1177–1186.

    Article  MathSciNet  MATH  Google Scholar 

  • Garthwaite, P.H., Kadane, J.B. and O’hagan, A. (2005). Statistical methods for eliciting probability distributions. J. Amer. Statist. Assoc., 100, 680–700.

    Article  MathSciNet  MATH  Google Scholar 

  • Gleser, L.J. (2002). [setting confidence intervals for bounded parameters]: Comment. Statist. Sci., 17, 161–163.

    Google Scholar 

  • Goldstein, M. (1997). Prior inferences for posterior judgements. In Structures and Norms in Science: Volume Two of the Tenth International Congress of Logic, Methodology and Philosophy of Science, Florence, August 1995 (M. L. D. Chiara, K. Doets, D. Mundici, & J. van Benthem Eds.). New York, Springer, pp. 55–71.

  • Goldstein, M. (2001). Avoiding foregone conclusions: Geometric and foundational analysis of paradoxes of finite additivity. J. Statist. Plann. Inference, 94, 73–87.

    Article  MathSciNet  MATH  Google Scholar 

  • Goldstein, M. (2006). Subjective Bayesian analysis: principles and practice. Bayesian Anal., 1, 403–420.

    Article  MathSciNet  Google Scholar 

  • Grundy, P.M. (1956). Fiducial distributions and prior distributions: an example in which the former cannot be associated with the latter. J. R. Stat. Soc. Ser. B, 18, 217–221.

    MathSciNet  MATH  Google Scholar 

  • Hacking, I. (1965). Logic of Statistical Inference. Cambridge University Press, Cambridge.

    MATH  Google Scholar 

  • Hacking, I. (2001). An introduction to probability and inductive logic. Cambridge University Press, Cambridge.

    Book  MATH  Google Scholar 

  • Hannig, J. (2009). On generalized fiducial inference. Statist. Sinica, 19, 491–544.

    MathSciNet  MATH  Google Scholar 

  • Hannig, J. and Xie, M. (2009). A note on Dempster-Shafer recombination of confidence distributions. Electron. J. Stat., 6, 1943–1966.

    Article  Google Scholar 

  • Heath, D. and Sudderth, W. (1978). On finitely additive priors, coherence, and extended admissibility. Ann. Statist., 6, 333–345.

    Article  MathSciNet  MATH  Google Scholar 

  • Heath, D. and Sudderth, W. (1989). Coherent inference from improper priors and from finitely additive priors. Ann. Statist., 17, 907–919.

    Article  MathSciNet  MATH  Google Scholar 

  • Helland, I.S. (2004). Statistical inference under symmetry. International Statistical Review, 72, 409–422.

    Article  Google Scholar 

  • Hurwicz, L. (1951). The generalized Bayes-minimax principle: a criterion for decision-making under uncertainty. Cowles Commission Discussion Paper 355.

  • Hwang, J.T., Casella, G., Robert, C., Wells, M.T. and Farrell, R.H. (1992). Estimation of accuracy in testing. Ann. Statist., 20, 490–509.

    Article  MathSciNet  MATH  Google Scholar 

  • Jaffray, J.-Y. (1989). Linear utility theory for belief functions. Oper. Res. Lett., 8, 107–112.

    Article  MathSciNet  MATH  Google Scholar 

  • Jeffrey, R. (1986). Probabilism and induction. Topoi, 5, 51–58.

    Article  MathSciNet  Google Scholar 

  • Kempthorne, O. (1976). Comment on E. T. Jaynes, ‘Confidence intervals vs Bayesian intervals’. Foundations of Probability Theory, Statistical Inference, and Statistical Theories of Science. D. Reidel, Dordrecht-Holland, Ch. Confidence intervals vs Bayesian intervals, pp. 220–228.

  • Kiefer, J. (1977). Conditional confidence statements and confidence estimators: rejoinder. J. Amer. Statist. Assoc., 72, 822–827.

    MathSciNet  Google Scholar 

  • Kohlas, J. and Monney, P.-A. (2008). An algebraic theory for statistical information based on the theory of hints. Internat. J. Approx. Reason., 48, 378–398.

    Article  MathSciNet  MATH  Google Scholar 

  • Kyburg, H.E. (2007). Probability and Inference. Texts in Philosophy 2. College Publications, London, Ch. Bayesian inference with evidential probability, pp. 281–296.

  • Kyburg, H.E. and Teng, C.M. (2001). Uncertain Inference. Cambridge University Press, Cambridge.

    Book  MATH  Google Scholar 

  • Lele, S.R. (2004). Elicit data, not prior: On using expert opinion in ecological studies. The Nature of Scientific Evidence: Statistical, Philosophical, and Empirical Considerations. University of Chicago Press, Chicago, pp. 410–436.

    Google Scholar 

  • Lindley, D.V. (1958). Fiducial distributions and Bayes’ theorem. J. R. Stat. Soc. Ser. B, 20, 102–107.

    MathSciNet  MATH  Google Scholar 

  • Liu, R. and Singh, K. (1997). Notions of limiting P values based on data depth and bootstrap. J. Amer. Statist. Assoc., 92, 266–277.

    MathSciNet  MATH  Google Scholar 

  • Maher, P. (1992). Diachronic rationality. Philos. Sci., 59, 120–141.

    Article  MathSciNet  Google Scholar 

  • Mccullagh, P. (2002). What is a statistical model? Ann. Statist., 30, 1225–1267.

    Article  MathSciNet  MATH  Google Scholar 

  • Paris, J.B. (1994). The Uncertain Reasoner’s Companion: A Mathematical Perspective. Cambridge University Press, New York.

    MATH  Google Scholar 

  • Polansky, A.M. (2007). Observed Confidence Levels: Theory and Application. Chapman and Hall, New York.

    Book  Google Scholar 

  • Robins, J. and Wasserman, L. (2000). Conditioning, likelihood, and coherence: A review of some foundational concepts. J. Amer. Statist. Assoc., 95, 1340–1346.

    Article  MathSciNet  MATH  Google Scholar 

  • Royall, R. (1997). Statistical Evidence: A Likelihood Paradigm. CRC Press, New York.

    MATH  Google Scholar 

  • Royall, R. (2000). On the probability of observing misleading statistical evidence. J. Amer. Statist. Assoc., 95, 760–768.

    Article  MathSciNet  MATH  Google Scholar 

  • Savage, L.J. (1954). The Foundations of Statistics. John Wiley and Sons, New York.

    MATH  Google Scholar 

  • Scheffe, H. (1977). A note on a reformulation of the s-method of multiple comparison. J. Amer. Statist. Assoc., 72, 143–146.

    MathSciNet  MATH  Google Scholar 

  • Schervish, M.J. (1995). Theory of Statistics. Springer-Verlag, New York.

    Book  MATH  Google Scholar 

  • Schweder, T., Hjort, N.L. (2002). Confidence and likelihood. Scand. J. Stat., 29, 309–332.

    Article  MathSciNet  MATH  Google Scholar 

  • Seidenfeld, T. (2007). Probability and Inference. Texts in Philosophy 2. College Publications, London, Ch. Forbidden fruit: When epistemological probability may not take a bite of the Bayesian apple, pp. 267–279.

  • Shafer, G. (2011). A betting interpretation for probabilities and Dempster-Shafer degrees of belief. Internat. J. Approx. Reason., 52, 127–136.

    Article  MathSciNet  MATH  Google Scholar 

  • Sharma, S.S. (1980). On hacking’s fiducial theory of inference. Canad. J. Statist., 8, 227–233.

    Article  MathSciNet  MATH  Google Scholar 

  • Singh, K., Xie, M. and Strawderman, W.E. (2005). Combining information from independent sources through confidence distributions. Ann. Statist., 33, 159–183.

    Article  MathSciNet  MATH  Google Scholar 

  • Sprott, D.A. (2000). Statistical Inference in Science. Springer, New York.

    MATH  Google Scholar 

  • Troffaes, M.C.M. (2007). Decision making under uncertainty using imprecise probabilities. Internat. J. Approx. Reason., 45, 17–29.

    Article  MathSciNet  MATH  Google Scholar 

  • Ville, J. (1939). Gauthier-Villars, Paris.

  • Vos, P. (2008). Boyles, R.A. (2008), “The role of likelihood in interval estimation,” The American Statistician, 62, 22–26: Comment by Vos and reply. Amer. Statist., 62, 274–275.

    Article  Google Scholar 

  • Wilkinson, G.N. (1977). On resolving the controversy in statistical inference (with discussion). J. R. Stat. Soc. Ser. B, 39, 119–171.

    MathSciNet  MATH  Google Scholar 

  • Williamson, J. (2009). Objective Bayesianism, Bayesian conditionalisation and voluntarism. Synthese, 178, 1–19.

    Google Scholar 

  • Zabell, S.L. (1992). R. A. Fisher and the fiducial argument. Statist. Sci., 7, 369–387.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgement.

I am grateful to the anonymous referee for comments that lead to several improvements in clarity and completeness, most notably the inclusion of Sections 3.2.2 and 5.2. Matthias Kohl kindly provided R code (R Development Core Team, 2004) used to compute the convolution of the double exponential distribution (“R-help” list message posted on 12/23/05). I thank Mark Cooper for helpful feedback and Jean Peccoud, Mark Whitsitt, Chris Martin, and Bob Merrill for their support of the seed of this paper (Bickel, 2006) at Pioneer Hi-Bred, International. Subsequent developments were partially supported by the Canada Foundation for Innovation and by the Ministry of Research and Innovation of Ontario.

Author information

Authors and Affiliations

  1. Ottawa Institute of Systems Biology, Department of Biochemistry, Microbiology and Immunology, Department of Mathematics and Statistics, University of Ottawa, 451 Smyth Road, Ottawa, ON, K1H 8M5, Canada

    David R. Bickel

Authors
  1. David R. Bickel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David R. Bickel.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bickel, D.R. A frequentist framework of inductive reasoning. Sankhya A 74, 141–169 (2012). https://doi.org/10.1007/s13171-012-0020-x

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13171-012-0020-x

Keywords and phrases.

AMS (2000) subject classification.

Search

Navigation