Theoretical Foundation for Statistical Analysis
This chapter introduces the reader to the theoretical foundations of statistical analysis by presenting a rigorous, multivariate, set-based approach to probability and statistical theory. The foundation is laid with consideration of the key statistical axioms and definitions, which formalise many of the concepts introduced in Chapter 1. Probability density functions, sample space, moments, the expectation operator, and various marginal functions are examined. Next, the most common statistical distributions, including the normal, Student’s t-, χ2-, F-, binomial, and Poisson distributions, are described by providing their key mathematical properties and computational implementation. Using these ideas, the subject of parameter estimation, that is, determining unknown values given a data set and an assumed model, is considered. Key topics include method of moments estimation, likelihood estimation, and regression estimation. Finally, the ability to compare two statistical variables using hypothesis testing and confidence intervals is introduced for many different commonly encountered cases, including means, variances, ratios, and paired values. Detailed examples are provided for all of the key concepts using simple, but relevant, examples. By the end of the chapter, the reader should have a strong understanding of the mathematical framework of statistics. As well, the ability to estimate parameters for a given situation and conduct appropriate hypothesis testing should be understood.