Simple Tests of Significance

Abstract

It is widely felt that psychologists and other behavioural scientists tend to over-emphasize the role of tests of significance in experimental work to the neglect of problems of estimation such as are discussed in chapter 4. To some extent the criticism is justified, but over-emphasis is largely a reflection of the nature of much of the data in question. Measurements of psychological phenomena, as noted earlier, tend to be imprecise and it is frequently the case that variation is considerable not only from subject to subject but for a single subject on different occasions. In view of this variation it is difficult to assess at a glance the performance of subjects on different occasions and in different experimental situations, whereas a test of significance, which takes the amount of variability in the data into account, provides a useful quantitative appraisal of the results. Perhaps the most important warning to the amateur is not to take the word ‘significance’ too literally. A difference between the mean scores of two samples of patients, which is ‘significant’ in the probability or statistical sense, may still not be of sufficient magnitude to be of any great importance in a clinical sense. In this chapter some simple tests for comparing pairs of means will be described and the distinction between statistical and clinical significance will be emphasized.