Regression Analysis

  • Ronald ChristensenEmail author
Part of the Springer Texts in Statistics book series (STS)


Francis Galton, a half-cousin of Charles Darwin, is often credited as the founder of regression analysis, a tool he used for studying heredity and the social sciences. R.A. Fisher seems to be responsible for our current focus of treating the X matrix as fixed and known. In this chapter we give a mathematical, rather than subject matter, definition of regression and discuss many of its standard features. We also introduce prediction theory which is based on having X random. We explore prediction theory’s close connection to regression and use it as the basis for defining many standard features of traditional regression.


  1. Aitchison, J., & Dunsmore, I. R. (1975). Statistical prediction analysis. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  2. Anderson, T. W. (2003). An introduction to multivariate statistical analysis (3rd ed.). New York: Wiley.zbMATHGoogle Scholar
  3. Atwood, C. L., & Ryan, T. A., Jr. (1977). A class of tests for lack of fit to a regression model. Unpublished manuscript.Google Scholar
  4. Box, G. E. P., & Draper, N. R. (1987). Empirical model-building and response surfaces. New York: Wiley.zbMATHGoogle Scholar
  5. Box, G. E. P., Hunter, W. G., & Hunter, J. S. (1978). Statistics for experimenters. New York: Wiley.zbMATHGoogle Scholar
  6. Brockwell, P. J., & Davis, R. A. (1991). Time series: Theory and methods (2nd ed.). New York: Springer.CrossRefGoogle Scholar
  7. Christensen, R. (1989). Lack of fit tests based on near or exact replicates. The Annals of Statistics, 17, 673–683.MathSciNetCrossRefGoogle Scholar
  8. Christensen, R. (1991). Small sample characterizations of near replicate lack of fit tests. Journal of the American Statistical Association, 86, 752–756.MathSciNetCrossRefGoogle Scholar
  9. Christensen, R. (1995). Comment on Inman (1994). The American Statistician, 49, 400.CrossRefGoogle Scholar
  10. Christensen, R. (2001). Advanced linear modeling: Multivariate, time series, and spatial data; nonparametric regression, and response surface maximization (2nd ed.). New York: Springer.CrossRefGoogle Scholar
  11. Christensen, R. (2015). Analysis of variance, design, and regression: Linear modeling for unbalanced data (2nd ed.). Boca Raton: Chapman and Hall/CRC Pres.Google Scholar
  12. Cook, R. D., & Weisberg, S. (1999). Applied regression including computing and graphics. New York: Wiley.CrossRefGoogle Scholar
  13. Cressie, N. (1993). Statistics for spatial data (Revised ed.). New York: Wiley.zbMATHGoogle Scholar
  14. Cressie, N. A. C., & Wikle, C. K. (2011). Statistics for spatio-temporal data. New York: Wiley.zbMATHGoogle Scholar
  15. Daniel, C., & Wood, F. S. (1980). Fitting equations to data (2nd ed.). New York: Wiley.zbMATHGoogle Scholar
  16. deLaubenfels, R. (2006). The victory of least squares and orthogonality in statistics. The American Statistician, 60, 315–321.MathSciNetCrossRefGoogle Scholar
  17. Doob, J. L. (1953). Stochastic processes. New York: Wiley.zbMATHGoogle Scholar
  18. Draper, N., & Smith, H. (1998). Applied regression analysis (3rd ed.). New York: Wiley.CrossRefGoogle Scholar
  19. Fisher, R. A. (1922). The goodness of fit of regression formulae, and the distribution of regression coefficients. Journal of the Royal Statistical Society, 85, 597–612.CrossRefGoogle Scholar
  20. Geisser, S. (1971). The inferential use of predictive distributions. In V.P. Godambe & D.A. Sprott (Eds.), Foundations of statistical inference. Toronto: Holt, Rinehart, and Winston.Google Scholar
  21. Geisser, S. (1993). Predictive inference: An introduction. New York: Chapman and Hall.CrossRefGoogle Scholar
  22. Gnanadesikan, R. (1977). Methods for statistical analysis of multivariate observations. New York: Wiley.zbMATHGoogle Scholar
  23. Hartigan, J. (1969). Linear Bayesian methods. Journal of the Royal Statistical Society, Series B, 31, 446–454.MathSciNetzbMATHGoogle Scholar
  24. Hinkley, D. V. (1969). Inference about the intersection in two-phase regression. Biometrika, 56, 495–504.CrossRefGoogle Scholar
  25. Hodges, J. S. (2013). Richly parameterized linear models: Additive, time series, and spatial models using random effects. Boca Raton: Chapman and Hall/CRC.zbMATHGoogle Scholar
  26. Johnson, R. A., & Wichern, D. W. (2007). Applied multivariate statistical analysis (6th ed.). Englewood Cliffs: Prentice-Hall.zbMATHGoogle Scholar
  27. McCulloch, C. E., Searle, S. R., & Neuhaus, J. M. (2008). Generalized, linear, and mixed models (2nd ed.). New York: Wiley.zbMATHGoogle Scholar
  28. Miller, F. R., Neill, J. W., & Sherfey, B. W. (1998). Maximin clusters for near replicate regression lack of fit tests. The Annals of Statistics, 26, 1411–1433.MathSciNetCrossRefGoogle Scholar
  29. Miller, F. R., Neill, J. W., & Sherfey, B. W. (1999). Implementation of maximin power clustering criterion to select near replicates for regression lack-of-fit tests. Journal of the American Statistical Association, 94, 610–620.MathSciNetCrossRefGoogle Scholar
  30. Morrison, D. F. (2004). Multivariate statistical methods (4th ed.). Pacific Grove: Duxbury Press.Google Scholar
  31. Neill, J. W., & Johnson, D. E. (1984). Testing for lack of fit in regression - a review. Communications in Statistics, Part A - Theory and Methods, 13, 485–511.MathSciNetCrossRefGoogle Scholar
  32. Pimentel, S. D., Small, D. S., & Rosenbaum, P. R. (2017). An exact test of fit for the Gaussian linear model using optimal nonbipartite matching. Technometrics, 59, 330–337.Google Scholar
  33. Rao, C. R. (1973). Linear statistical inference and its applications (2nd ed.). New York: Wiley.CrossRefGoogle Scholar
  34. Ripley, B. D. (1981). Spatial statistics. New York: Wiley.CrossRefGoogle Scholar
  35. Shillington, E. R. (1979). Testing lack of fit in regression without replication. Canadian Journal of Statistics, 7, 137–146.CrossRefGoogle Scholar
  36. Shumway, R. H., & Stoffer, D. S. (2011). Time series analysis and its applications: With R examples (3rd ed.). New York: Springer.CrossRefGoogle Scholar
  37. Smith, A. F. M. (1986). Comment on an article by B. Efron. The American Statistician, 40, 10.Google Scholar
  38. Snedecor, G. W., & Cochran, W. G. (1980). Statistical methods (7th ed.). Ames: Iowa State University Press.zbMATHGoogle Scholar
  39. Utts, J. (1982). The rainbow test for lack of fit in regression. Communications in Statistics—Theory and Methods, 11, 2801–2815.MathSciNetCrossRefGoogle Scholar
  40. Weisberg, S. (2014). Applied linear regression (4th ed.). New York: Wiley.zbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Mathematics and StatisticsUniversity of New MexicoAlbuquerqueUSA

Personalised recommendations