Significance, truth and proof of p values: reminders about common misconceptions regarding null hypothesis significance testing
- 1.5k Downloads
Null hypothesis significance testing has successfully reduced the complexity of scientific inference to a dichotomous decision (i.e., ‘reject’ versus ‘not reject’). As a consequence, p values and their associated statistical significance play an important role in the social and medical sciences. But do we truly understand what statistical significance testing and p values entail? Judging by the vast literature on controversies regarding their application and interpretation, this seems questionable. It has even been argued that significance testing should be abandoned all together . We seek to extend Fayer’s  paper on statistically significant correlations and to clarify some of the controversies regarding statistical significance testing by explaining that (1) the pvalue is not the probability of the null hypothesis; (2) rejecting the null hypothesis does not prove that the alternative hypothesis is true; (3) not rejecting the null hypothesis does not prove that the alternative...