Conducting Multiple Comparisons

Abstract

Previous chapters described how to choose and combine different independent ­variables and their conditions. When there are more than two conditions for these variables, additional statistical analysis is needed to verify which differences between conditions are significant. This chapter describes the steps that need to be taken when there are more than two conditions that are being compared with follow-up statistical tests.

Both t-tests and ANOVA can be used to conduct multiple comparisons. When t-tests are conducted, each test indicates whether a difference between two conditions is significant. However, each test carries its own risk. As a result, making multiple comparisons increases the chance of making a Type I Error - where a null hypothesis is incorrectly rejected. To avoid increasing the overall chance of making a Type I Error, a Bonferroni adjustment should be made. On the other hand, when ANOVA is conducted the omnibus F-test shows main and interaction effects. When there are more than two conditions, these significant effects by themselves do not indicate which pairs of conditions differ from each other and follow-up analysis is needed. With ANOVA, the additional analysis consists of post hoc comparisons to pinpoint the conditions where the differences are significant.