Abstract
Imagine that you conduct an experiment to determine whether a new form of psychotherapy is more effective than treatment as usual. You analyze the data and find them to be statistically significant with a p-value that is less than 5%. How accurately does the following paragraph describe your understanding of the meaning of these results?
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66, 1–29.
Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304–1312.
Cohen, J. (1994). The earth is round (p<.05). American Psychologist, 49, 997–1003.
Lykken, D. E. (1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151–159.
Rosnow, R. L. & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276–84.
Rozeboom, W. W. (1960). The fallacy of the null hypothesis significance test. Psychological Bulletin, 57, 416–428.
Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 69, 145–153.
Sedlmeier, P. & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies. Psychological Bulletin, 105, 309–316.
Zuckerman, M., Hodgkins, H. S., Zuckerman, A., & Rosenthal, R. (1993). Contemporary issues in the analysis of data: A survey of 551 psychologists. Psychological Science, 4, 49–53.
Rosenthal, R. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638–641.
Kirsch, I., Moore, T. J., Scoboria, A., & Nicholls, S. S. (2002). The emperor’s new drugs: An analysis of antidepressant medication data submitted to the U.S. Food and Drug Administration. Prevention and Treatment, 5, Article 23, available at http://content.apa.org/journals/pre/5/1/23.
McFall, R. M. (1991). Manifesto for a science of clinical psychology. American Psychologist, 44, 75–88.
Weston, D., Novotny, C. M., & Thompson-Brenner, H. (2004). The empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controlled clinical trials. Psychological Bulletin, 130, 631–663.
Luborsky, L., Diguer, L., Seligman, D. A., Rosenthal, R., Krause, E. D., Johnson, S., et al. (1999). The researcher’s own therapy allegiances: A “wild card” in comparisons of treatment efficacy. Clinical Psychology. Science and Practice, 6, 95–106.
Atkins, D. C., Bedics, J. D., McGlinchey, J. B., & Beauchaine, T. P. (2005). Assessing clinical significance: Does it matter which method we use. Journal of Consulting and Clinical Psychology, 73, 982–989.
Jacobson, N. S. & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19.
Cohen, J. (1988). Statistical power analysis for the behavioral science, 2nd edn. Hillsdale, NJ: Erlbaum.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Humana Press, a part of Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Kelley, J.M. (2009). The Perils of p-Values: Why Tests of Statistical Significance Impede the Progress of Research. In: Levy, R.A., Ablon, J.S. (eds) Handbook of Evidence-Based Psychodynamic Psychotherapy. Current Clinical Psychiatry. Humana Press. https://doi.org/10.1007/978-1-59745-444-5_16
Download citation
DOI: https://doi.org/10.1007/978-1-59745-444-5_16
Publisher Name: Humana Press
Print ISBN: 978-1-934115-11-4
Online ISBN: 978-1-59745-444-5
eBook Packages: MedicineMedicine (R0)