Abstract
This chapter revisits the significance test controversy in the light of Jeffreys’ views about the role of statistical inference in experimental investigations. These views have been clearly expressed in the third edition of his Theory of Probability. The relevant passage is quoted and commented. The elementary inference about the difference between two means is considered, but the conclusions are applicable to most of the usual situations encountered in experimental data analysis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
American Psychological Association: Publication Manual of the American Psychological Association, 6th edn. American Psychological Association, Washington (2010)
Bakan, D.: The test of significance in psychological research. Psychol. Bull. 66, 423–437 (1966)
Berger, J.: The case for objective Bayesian analysis. Bayesian Anal. 11, 1–17 (2004)
Box, G.E.P., Tiao, G.C.: Bayesian Inference in Statistical Analysis. Addison Wesley, Reading (1973)
Bunouf, P., Lecoutre, B.: Bayesian priors in sequential binomial design. Cr. Acad. Sci. I-Math. 343, 339–344 (2006)
Bunouf, P., Lecoutre, B.: An objective Bayesian approach to multistage hypothesis testing. Sequential Anal. 29, 88–101 (2010)
Carver, R.P.: The case against statistical significance testing. Harvard Educ. Rev. 48, 378–399 (1978)
Casella, G., Berger, L.: Reconciling Bayesian and frequentist evidence in the one-sided testing problem. J. Am. Stat. Assoc. 82, 106–111 (1987)
Cochran, W.G., Cox, G.M.: Experimental Designs, 2nd edn. Wiley, New York (1957)
Cox, D.R.: Another comment on the role of statistical methods. Brit. Med. J. 322, 231 (2001)
de Cristofaro, R.: On the foundations of likelihood principle. J. Stat. Plan. Infer. 126, 401–411 (2004)
de Cristofaro, R.: Foundations of the ‘Objective Bayesian Inference’. In: First Symposium on Philosophy, History and Methodology of ERROR. Virginia Tech, Blacksburg (2006). http://www.error06.econ.vt.edu/Christofaro.pdf. Accessed 13 March 2014
Edwards, W., Lindman, H., Savage, L.J.: Bayesian statistical inference for psychological research. Psychol. Rev. 70, 193–242 (1963)
Fraser, D.A.S.: Discussion of Hill’s paper, on some statistical paradoxes and non-conglomerability. Trab. Estad. Investig. Oper. 31, 56–58 (1980)
Geisser, S.: Predictive Inference: An Introduction. Chapman & Hall, New York (1993)
Grouin, J.-M., Coste, M., Bunouf, P., Lecoutre, B.: Bayesian sample size determination in non-sequential clinical trials: Statistical aspects and some regulatory considerations. Stat. Med. 26(26), 4914–4924 (2007)
ICH E9 Expert Working Group: Statistical principles for clinical trials: ICH harmonised tripartite guideline, current step 4 version. http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E9/Step4/E9_Guideline.pdf (1998). Cited 13 Mar 2014
Iverson, G.J., Lee, M.D., Wagenmakers, E.-J.: \(p_{ \text{ riptsizerep }}\) misestimates the probability of replication. Psychon. B. Rev. 16, 424–429 (2009)
Jaynes, E.T.: Confidence intervals vs Bayesian intervals (with discussion). In: Harper, W.L., Hooker, C.A. (eds.) Statistical Inference and Statistical Theories of Science, vol. 2, pp. 175–257. D. Reidel, Dordrecht, The Netherlands (1976)
Jeffreys, H., Jeffreys, H.: Theory of Probability, 3rd edn. Clarendon, Oxford (1967). (1st edn 1939)
Jones, L.V.: Statistics and research design. Annu. Rev. Psychol. 6, 405–430 (1995)
Jones, L.V., Tukey, J.W.: A sensible formulation of the significance test. Psychol. Methods 5, 411–414 (2000)
Killeen, P.R.: An alternative to null-hypothesis significance tests. Psychol. Sci. 16, 345–353 (2005)
Kruschke, J.K.: Bayesian assessment of null values via parameter estimation and model comparison. Perspect. Psychol. Sci. 6, 299–312 (2011)
Lecoutre, B., Lecoutre, M.-P., Poitevineau, J.: Killeen’s probability of replication and predictive probabilities: How to compute, use and interpret them. Psychol. Methods 15, 158–171 (2010)
Lecoutre, B., Killeen, P.: Replication is not coincidence: Reply to Iverson, Lee, and Wagenmakers (2009). Psychon. B. Rev. 17, 263–269 (2010)
Matloff, N.S.: Statistical hypothesis testing: problems and alternatives. Environ. Entomol. 20, 1246–1250 (1991)
Meehl, P.E.: Theory testing in psychology and physics: A methodological paradox. Philos. Sci. 34, 103–115 (1967)
Meehl, P.E.: Appraising and amending theories: the strategy of Lakatosian defense and two principles that warrant it. Psychol. Inq. 1, 108–141 (1990)
Morrison, D.E., Henkel, R.E. (eds.): The Significance Test Controversy–A Reader. Butterworths, London (1970)
Pearson, E.S.: “Student” as statisticianAuthor. Biometrika 30, 210–250 (1939)
Robert, C.P.: The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation. Springer, New York (2007)
Robert, C.P., Chopin, N., Rousseau, J.: Harold Jeffreys’s theory of probability revisited (with comments). Stat. Sci. 24, 141–194 (2009)
Rosenthal, R., Rubin, D.B.: The counternull value of an effect size: A new statistic. Psychol. Sci. 5, 329–334 (1994)
Rozeboom, W.W.: The fallacy of the null hypothesis significance test. Psychol. Bull. 57, 416–428 (1960)
Sterne, J.A.C., Smith, G.D.: Sifting the evidence-what’s wrong with significance tests? Brit. Med. J. 322, 226–231 (2001)
Student, : The probable error of a mean. Biometrika 6(1), 1–25 (1908)
Student, : Tables for estimating the probability that the mean of a unique sample of observations lies between \(-\infty \) and any given distance of the mean of the population from which the sample is drawn. Biometrika 11, 414–417 (1917)
Tukey, J.W.: Conclusions vs decisions. Technometrics 2, 1–11 (1960)
Tukey, J.W.: The philosophy of multiple comparisons. Stat. Sci. 6, 100–116 (1991)
Yates, F.: Sir Ronald Fisher and the design of experiments. Biometrics 20, 307–321 (1964)
Zabell, S.L.: On Student’s 1908 paper “The probable error of a mean”. J. Am. Stat. Assoc. 103, 1–7 (2008)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2014 The Author(s)
About this chapter
Cite this chapter
Lecoutre, B., Poitevineau, J. (2014). The Significance Test Controversy Revisited. In: The Significance Test Controversy Revisited. SpringerBriefs in Statistics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44046-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-662-44046-9_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44045-2
Online ISBN: 978-3-662-44046-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)