Abstract
Based on the banning of null hypothesis significance testing and confidence intervals in Basic and Applied Psychology (2015), this presentation focusses on alternative ways for researchers to think about inference. One section reviews literature on the a priori procedure. The basic idea, here, is that researchers can perform much inferential work before the experiment. Furthermore, this possibility changes the scientific philosophy in important ways. A second section moves to what researchers should do after they have collected their data, with an accent on obtaining a better understanding of the obtained variance. Researchers should try out a variety of summary statistics, instead of just one type (such as means), because seemingly conceptually similar summary statistics nevertheless can imply very different qualitative stories. Also, rather than engage in the typical bipartite distinction between variance due to the independent variable and variance not due to the independent variable; a tripartite distinction is possible that divides variance not due to the independent variable into variance due to systematic or random factors, with important positive consequences for researchers. Finally, the third major section focusses on how researchers should or should not draw causal conclusions from their data. This section features a discussion of within-participants causation versus between-participants causation, with an accent on whether the type of causation specified in the theory is matched or mismatched by the type of causation tested in the experiment. There also is a discussion of causal modeling approaches, with criticisms. The upshot is that researchers could do much more a priori work, and much more a posteriori work too, to maximize the scientific gains they obtain from their empirical research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Nguyen (2016) provided an informative theoretical perspective on the ban.
- 2.
Of course, the null hypothesis significance testing procedure does not test the hypothesis of interest but rather the null hypothesis that is not of interest, which is one of the many criticisms to which the procedure has been subjected. But as the present focus is on what to do instead, I will not focus on these criticisms. The interested reader can consult Trafimow and Earp (2017).
- 3.
In addition, \( \omega \) is of more interest than \( \sigma \) though this is not of great importance yet.
- 4.
The reader may wonder why skewness increases precision. For a quantitative answer, see Trafimow et al. (in press). For a qualitative answer, simply look up pictures of skew-normal distributions (contained in Trafimow et al., among other places). Observe that as the absolute magnitude of skewness increases, the bulk of the distributions become taller and narrower. Hence, sampling precision increases.
- 5.
For skew-normal distributions it makes more sense to consider the square of the scale than to consider the square of the standard deviation, known as the variance. But researchers are used to variance and variance is sufficient to make the necessary points in this section.
- 6.
I provide all the equations necessary to calculate the adjusted success rate in Trafimow (2017b).
References
Blanca, M.J., Arnau, J., López-Montiel, D., Bono, R., Bendayan, R.: Skewness and kurtosis in real data samples. Methodol. Eur. J. Res. Methods Behav. Soc. Sci. 9(2), 78–84 (2013)
Cain, M.K., Zhang, Z., Yuan, K.H.: Behav. Res. Methods 49(5), 1716–1735 (2017)
Earp, B.D., Trafimow, D.: Replication, falsification, and the crisis of confidence in social psychology. Front. Psychol. 6(621), 1–11 (2015)
Fishbein, M., Ajzen, I.: Predicting and changing behavior: The Reasoned Action Approach. Psychology Press (Taylor & Francis), New York (2010)
Gillies, D.: Philosophical theories of probability. Routledge, London (2000)
Grice, J.W., Cohn, A., Ramsey, R.R., Chaney, J.M.: On muddled reasoning and mediation modeling. Basic Appl. Soc. Psychol. 37(4), 214–225 (2015)
Gulliksen, H.: Theory of Mental Tests. Lawrence Erlbaum Associates Publishers, Hillsdale (1987)
Ho, A.D., Yu, C.C.: Descriptive statistics for modern test score distributions: Skewness, kurtosis, discreteness, and ceiling effects. Educ. Psychol. Measur. 75(3), 365–388 (2015)
Kline, R.B.: The mediation myth. Basic Appl. Soc. Psychol. 37(4), 202–213 (2015)
Lord, F.M., Novick, M.R.: Statistical theories of mental test scores. Addison-Wesley, Reading (1968)
Micceri, T.: The unicorn, the normal curve, and other improbable creatures. Psychol. Bull. 105(1), 156–166 (1989)
Nguyen, H.T.: On evidential measures of support for reasoning with integrated uncertainty: a lesson from the ban of P-values in statistical inference. In: Huynh, V.N. et al. (Eds.) Integrated Uncertainty in Knowledge Modeling and Decision Making, Lecture notes in Artificial Intelligence, vol, 9978, pp. 3–15. Springer, Cham (2016)
Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search. The MIT Press, Cambridge (2000)
Tate, C.U.: On the overuse and misuse of mediation analysis: it may be a matter of timing. Basic Appl. Soc. Psychol. 37(4), 235–246 (2015)
Thoemmes, F.: Reversing arrows in mediation models does not distinguish plausible models. Basic Appl. Soc. Psychol. 37(4), 226–234 (2015)
Trafimow, D.: Editorial. Basic Appl. Soc. Psychol. 36(1), 1–2 (2014)
Trafimow, D.: Introduction to special issue: what if planetary scientists used mediation analysis to infer causation? Basic Appl. Soc. Psychol. 37(4), 197–201 (2015)
Trafimow, D.: Using the coefficient of confidence to make the philosophical switch from a posteriori to a priori inferential statistics. Educ. Psychol. Measur. 77(5), 831–854 (2017a)
Trafimow, D.: Comparing the descriptive characteristics of the adjusted success rate to the phi coefficient, the odds ratio, and the difference between conditional proportions. Int. J. Stat. Adv. Theory Appl. 1(1), 1–19 (2017b)
Trafimow, D.: The probability of simple versus complex causal models in causal analyses. Behav. Res. Methods 49(2), 739–746 (2017c)
Trafimow, D.: Some implications of distinguishing between unexplained variance that is systematic or random. Educ. Psychol. Measur. 78(3), 482–503 (2018)
Trafimow, D.: My ban on null hypothesis significance testing and confidence intervals. Studies in Computational Intelligence (in press a)
Trafimow, D.: An a priori solution to the replication crisis. Philos. Psychol. 31(8), 1188–1214 (2018)
Trafimow, D., Amrhein, V., Areshenkoff, C.N., Barrera-Causil, C.J., Beh, E.J., Bilgiç, Y.K., Bono, R., Bradley, M.T., Briggs, W.M., Cepeda-Freyre, H.A., Chaigneau, S.E., Ciocca, D.R., Correa, J.C., Cousineau, D., de Boer, M.R., Dhar, S.S., Dolgov, I., Gómez-Benito, J., Grendar, M., Grice, J.W., Guerrero-Gimenez, M.E., Gutiérrez, A., Huedo-Medina, T.B., Jaffe, K., Janyan, A., Karimnezhad, A., Korner-Nievergelt, F., Kosugi, K., Lachmair, M., Ledesma, R.D., Limongi, R., Liuzza, M.T., Lombardo, R., Marks, M.J., Meinlschmidt, G., Nalborczyk, L., Nguyen, H.T., Ospina, R., Perezgonzalez, J.D., Pfister, R., Rahona, J.J., Rodríguez-Medina, D.A., Romão, X., Ruiz-Fernández, S., Suarez, I., Tegethoff, M., Tejo, M., van de Schoot, R., Vankov, I.I., Velasco-Forero, S., Wang, T., Yamada, Y., Zoppino, F.C.M., Marmolejo-Ramos, F.: Manipulating the alpha level cannot cure significance testing. Front. Psychol. 9, 699 (2018a)
Trafimow, D., Clayton, K.D., Sheeran, P., Darwish, A.-F.E., Brown, J.: How do people form behavioral intentions when others have the power to determine social consequences? J. Gen. Psychol. 137, 287–309 (2010)
Trafimow, D., Kiekel, P.A., Clason, D.: The simultaneous consideration of between-participants and within-participants analyses in research on predictors of behaviors: the issue of dependence. Eur. J. Soc. Psychol. 34, 703–711 (2004)
Trafimow, D., MacDonald, J.A.: Performing inferential statistics prior to data collection. Educ. Psychol. Measur. 77(2), 204–219 (2017)
Trafimow, D., Marks, M.: Editorial. Basic Appl. Soc. Psychol. 37(1), 1–2 (2015)
Trafimow, D., Marks, M.: Editorial. Basic Appl. Soc. Psychol. 38(1), 1–2 (2016)
Trafimow, D., Wang, T., Wang, C.: Means and standard deviations, or locations and scales? That is the question! New Ideas Psychol. 50, 34–37 (2018b)
Trafimow, D., Wang, T., Wang, C.: From a sampling precision perspective, skewness is a friend and not an enemy! Educ. Psychol. Meas. (in press)
Trueblood, J.S., Busemeyer, J.R.: A quantum probability account of order effects in inference. Cogn. Sci. 35, 1518–1552 (2011)
Trueblood, J.S., Busemeyer, J.R.: A quantum probability model of causal reasoning. Front. Psychol. 3, 138 (2012)
Valentine, J.C., Aloe, A.M., Lau, T.S.: Life after NHST: How to describe your data without “p-ing” everywhere. Basic Appl. Soc. Psychol. 37(5), 260–273 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Trafimow, D. (2019). What to Do Instead of Null Hypothesis Significance Testing or Confidence Intervals. In: Kreinovich, V., Thach, N., Trung, N., Van Thanh, D. (eds) Beyond Traditional Probabilistic Methods in Economics. ECONVN 2019. Studies in Computational Intelligence, vol 809. Springer, Cham. https://doi.org/10.1007/978-3-030-04200-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-04200-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04199-1
Online ISBN: 978-3-030-04200-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)