Statistical Analysis of Experimental Data

Abstract

While an increasing number of observational studies in modern political science use quite sophisticated statistical methods, experimental studies often continue to apply rather simple statistical instruments like t-tests or analysis of variance (ANOVA). At first sight this is surprising if one considers that many modern statistical methods had been developed and invented for the analysis of data generated in random experiments (see, for example, Fisher, 1935). It is, however, less surprising if one considers that more sophisticated statistical methods were developed much later in order to cope with specific data problems in observational studies. Looking from the perspective of the random experimentalist, the most serious statistical challenges in observational data arise from treatment imbalance and from the violation of the assumption that the independent variables are distributed independently and identically at random.