LMs, GLMs and GAMs

  • Thomas W. Yee
Part of the Springer Series in Statistics book series (SSS)


For completeness, this chapter summarizes some relevant aspects of the linear model (LM) and generalized linear model (GLM) for the book. A basic understanding of these is helpful when considering VGLMs later. Some topics covered include link functions, the exponential family, assumptions, estimation (especially IRLS), numerical and computing aspects, how to fit these models in R, and diagnostics. Also, a section on smoothing is provided, which covers splines and local regression at an introductory level. Generalized additive models are briefly mentioned.


  1. Agresti, A. 2015. Foundations of Linear and Generalized Linear Models. Hoboken: Wiley.zbMATHGoogle Scholar
  2. Aitkin, M., B. Francis, J. Hinde, and R. Darnell 2009. Statistical Modelling in R. Oxford: Oxford University Press.zbMATHGoogle Scholar
  3. Albert, A. and J. A. Anderson 1984. On the existence of maximum likelihood estimates in logistic regression models. Biometrika 71(1):1–10.zbMATHMathSciNetCrossRefGoogle Scholar
  4. Allison, P. 2004. Convergence problems in logistic regression. See Altman et al. (2004), pp. 238–252.Google Scholar
  5. Altman, M., J. Gill, and M. P. McDonald 2004. Numerical Issues in Statistical Computing for the Social Scientist. Hoboken: Wiley-Interscience.zbMATHGoogle Scholar
  6. Amemiya, T. 1985. Advanced Econometrics. Oxford: Blackwell.Google Scholar
  7. Azzalini, A. 1996. Statistical Inference: Based on the Likelihood. London: Chapman & Hall.zbMATHGoogle Scholar
  8. Banerjee, S. and A. Roy 2014. Linear Algebra and Matrix Analysis for Statistics. Boca Raton: CRC Press.zbMATHGoogle Scholar
  9. Bellman, R. E. 1961. Adaptive Control Processes. Princeton: Princeton University Press.zbMATHGoogle Scholar
  10. Belsley, D. A., E. Kuh, and R. E. Welsch 1980. Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. New York: John Wiley & Sons.zbMATHCrossRefGoogle Scholar
  11. Berlinet, A. and C. Thomas-Agnan 2004. Reproducing Kernel Hilbert Spaces in Probability and Statistics. Boston: Kluwer Academic Publishers.zbMATHCrossRefGoogle Scholar
  12. Buja, A., T. Hastie, and R. Tibshirani 1989. Linear smoothers and additive models. The Annals of Statistics 17(2):453–510. With discussion.Google Scholar
  13. Christensen, R. 2011. Plane Answers to Complex Questions: The Theory of Linear Models (4th ed.). New York: Springer-Verlag.CrossRefGoogle Scholar
  14. Cleveland, W. S. 1979. Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association 74(368):829–836.zbMATHMathSciNetCrossRefGoogle Scholar
  15. Cleveland, W. S. and S. J. Devlin 1988. Locally weighted regression: An approach to regression analysis by local fitting. Journal of the American Statistical Association 83(403):596–610.zbMATHCrossRefGoogle Scholar
  16. Cleveland, W. S., E. Grosse, and W. M. Shyu 1991. Local regression models. See Chambers and Hastie (1991), pp. 309–376.Google Scholar
  17. Cook, R. D. and S. Weisberg 1982. Residuals and Influence in Regression. Monographs on Statistics and Applied Probability. London: Chapman & Hall.Google Scholar
  18. de Boor, C. 2001. A Practical Guide to Splines (Revised Edition). New York: Springer.Google Scholar
  19. Dobson, A. J. and A. Barnett 2008. An Introduction to Generalized Linear Models (Third ed.). Boca Raton: Chapman & Hall/CRC Press.zbMATHGoogle Scholar
  20. Eilers, P. H. C. and B. D. Marx 1996. Flexible smoothing with B-splines and penalties. Statistical Science 11(2):89–121.zbMATHMathSciNetCrossRefGoogle Scholar
  21. Eubank, R. L. 1999. Spline Smoothing and Nonparametric Regression (Second ed.). New York: Marcel-Dekker.zbMATHGoogle Scholar
  22. Fahrmeir, L. and G. Tutz 2001. Multivariate Statistical Modelling Based on Generalized Linear Models (Second ed.). New York: Springer-Verlag.zbMATHCrossRefGoogle Scholar
  23. Fan, J. and I. Gijbels 1996. Local Polynomial Modelling and Its Applications. London: Chapman & Hall.zbMATHGoogle Scholar
  24. Faraway, J. J. 2006. Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models. Boca Raton: Chapman and Hall/CRC.Google Scholar
  25. Faraway, J. J. 2015. Linear Models with R (Second ed.). Boca Raton: Chapman & Hall/CRC.zbMATHGoogle Scholar
  26. Fessler, J. A. 1991. Nonparametric fixed-interval smoothing with vector splines. IEEE Transactions on Signal Processing 39(4):852–859.CrossRefGoogle Scholar
  27. Firth, D. 1991. Generalized linear models. See Hinkley et al. (1991), pp. 55–82.Google Scholar
  28. Fox, J. and S. Weisberg 2011. An R Companion to Applied Regression (Second ed.). Thousand Oaks: Sage Publications.Google Scholar
  29. Gentle, J. E., W. K. Härdle, and Y. Mori 2012. Handbook of Computational Statistics: Concepts and Methods (Second ed.). Berlin: Springer.CrossRefGoogle Scholar
  30. Green, P. J. 1984. Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives. Journal of the Royal Statistical Society, Series B 46(2):149–192. With discussion.Google Scholar
  31. Green, P. J. and B. W. Silverman 1994. Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach. London: Chapman & Hall.zbMATHCrossRefGoogle Scholar
  32. Gu, C. 2013. Smoothing Spline ANOVA Models (Second ed.). New York, USA: Springer.zbMATHCrossRefGoogle Scholar
  33. Härdle, W. 1987. Smoothing Techniques With Implementation in S. New York, USA: Springer-Verlag.Google Scholar
  34. Härdle, W. 1990. Applied Nonparametric Regression. Cambridge: Cambridge University Press.zbMATHCrossRefGoogle Scholar
  35. Härdle, W., H. Liang, and J. Gao 2000. Partially Linear Models. New York, USA: Springer.zbMATHCrossRefGoogle Scholar
  36. Härdle, W., M. Müller, S. Sperlich, and A. Werwatz 2004. Nonparametric and Semiparametric Models. Berlin: Springer.zbMATHCrossRefGoogle Scholar
  37. Harezlak, J., D. Ruppert, and M.P. Wand. 2016. Semiparametric regression in R. New York: Springer.Google Scholar
  38. Harville, D. A. 1997. Matrix Algebra From a Statistician’s Perspective. New York, USA: Springer-Verlag.zbMATHCrossRefGoogle Scholar
  39. Hastie, T. 1996. Pseudosplines. Journal of the Royal Statistical Society, Series B 58(2):379–396.zbMATHMathSciNetGoogle Scholar
  40. Hastie, T. J. and D. Pregibon 1991. Generalized linear models. See Chambers and Hastie (1991), pp. 195–247.Google Scholar
  41. Hastie, T. J. and R. J. Tibshirani 1990. Generalized Additive Models. London: Chapman & Hall.zbMATHGoogle Scholar
  42. Hastie, T. J., R. J. Tibshirani, and J. H. Friedman 2009. The Elements of Statistical Learning: Data Mining, Inference and Prediction (Second ed.). New York, USA: Springer-Verlag.CrossRefGoogle Scholar
  43. Hauck, J. W. W. and A. Donner 1977. Wald’s test as applied to hypotheses in logit analysis. Journal of the American Statistical Association 72(360):851–853.zbMATHMathSciNetCrossRefGoogle Scholar
  44. Heinze, G. and M. Schemper 2002. A solution to the problem of separation in logistic regression. Statistics in Medicine 21(16):2409–2419.CrossRefGoogle Scholar
  45. James, G., D. Witten, T. Hastie, and R. Tibshirani 2013. An Introduction to Statistical Learning with Applications in R. New York, USA: Springer.zbMATHCrossRefGoogle Scholar
  46. Kohn, R. and C. F. Ansley 1987. A new algorithm for spline smoothing based on smoothing a stochastic process. SIAM Journal on Scientific and Statistical Computing 8(1):33–48.zbMATHMathSciNetCrossRefGoogle Scholar
  47. Lindsey, J. K. 1997. Applying Generalized Linear Models. New York, USA: Springer-Verlag.zbMATHGoogle Scholar
  48. Loader, C. 1999. Local Regression and Likelihood. New York, USA: Springer.zbMATHGoogle Scholar
  49. Marra, G. and R. Radice 2010. Penalised regression splines: theory and application to medical research. Statistical Methods in Medical Research 19(2):107–125.MathSciNetCrossRefGoogle Scholar
  50. McCullagh, P. and J. A. Nelder 1989. Generalized Linear Models (Second ed.). London: Chapman & Hall.zbMATHCrossRefGoogle Scholar
  51. Myers, R. H., D. C. Montgomery, G. G. Vining, and T. J. Robinson 2010. Generalized Linear Models With Applications in Engineering and the Sciences (Second ed.). Hoboken, NJ, USA: Wiley.zbMATHCrossRefGoogle Scholar
  52. Nelder, J. A. and R. W. M. Wedderburn 1972. Generalized linear models. Journal of the Royal Statistical Society, Series A 135(3):370–384.CrossRefGoogle Scholar
  53. Nosedal-Sanchez, A., C. B. Storlie, T. C. M. Lee, and R. Christensen 2012. Reproducing kernel Hilbert spaces for penalized regression: A tutorial. American Statistician 66(1):50–60.MathSciNetCrossRefGoogle Scholar
  54. Rao, C. R. 1973. Linear Statistical Inference and its Applications (Second ed.). New York, USA: Wiley.zbMATHCrossRefGoogle Scholar
  55. Reinsch, C. H. 1967. Smoothing by spline functions. Numerische Mathematik 10(3):177–183.zbMATHMathSciNetCrossRefGoogle Scholar
  56. Rencher, A. C. and G. B. Schaalje 2008. Linear Models in Statistics (second ed.). New York, USA: John Wiley & Sons.zbMATHGoogle Scholar
  57. Ruppert, D., M. P. Wand, and R. J. Carroll 2003. Semiparametric Regression. Cambridge: Cambridge University Press.zbMATHCrossRefGoogle Scholar
  58. Ruppert, D., M. P. Wand, and R. J. Carroll 2009. Semiparametric regression during 2003–2007. Electronic Journal of Statistics 3(1):1193–1256.zbMATHMathSciNetCrossRefGoogle Scholar
  59. Schimek, M. G. (Ed.) 2000. Smoothing and Regression: Approaches, Computation, and Application. New York, USA: Wiley.Google Scholar
  60. Schumaker, L. L. 2007. Spline Functions: Basic Theory (Third ed.). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  61. Seber, G. A. F. 2008. A Matrix Handbook for Statisticians. Hoboken, NJ, USA: Wiley.Google Scholar
  62. Seber, G. A. F. and A. J. Lee 2003. Linear Regression Analysis (Second ed.). New York, USA: Wiley.zbMATHCrossRefGoogle Scholar
  63. Silverman, B. W. 1984. Spline smoothing: The equivalent variable kernel method. The Annals of Statistics 12(3):898–916.zbMATHMathSciNetCrossRefGoogle Scholar
  64. Silverman, B. W. 1985. Some aspects of the spline smoothing approach to nonparametric regression curve fitting. Journal of the Royal Statistical Society, Series B 47(1):1–21. With discussion.Google Scholar
  65. Venables, W. N. and B. D. Ripley 2002. Modern Applied Statistics With S (4th ed.). New York, USA: Springer-Verlag.zbMATHCrossRefGoogle Scholar
  66. Wahba, G. 1990. Spline models for observational data, Volume 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. Philadelphia, PA, USA: Society for Industrial and Applied Mathematics (SIAM).Google Scholar
  67. Wand, M. P. and M. C. Jones 1995. Kernel Smoothing. London: Chapman & Hall.zbMATHCrossRefGoogle Scholar
  68. Wand, M. P. and J. T. Ormerod 2008. On semiparametric regression with O’Sullivan penalized splines. Australian & New Zealand Journal of Statistics 50(2):179–198.zbMATHMathSciNetCrossRefGoogle Scholar
  69. Wang, Y. 2011. Smoothing Splines: Methods and Applications. Boca Raton, FL, USA: Chapman & Hall/CRC.Google Scholar
  70. Wecker, W. E. and C. F. Ansley 1983. The signal extraction approach to nonlinear regression and spline smoothing. Journal of the American Statistical Association 78(381):81–89.zbMATHMathSciNetCrossRefGoogle Scholar
  71. Wedderburn, R. W. M. 1974. Quasi-likelihood functions, generalized linear models, and the Gauss-Newton method. Biometrika 61(3):439–447.zbMATHMathSciNetGoogle Scholar
  72. Wold, S. 1974. Spline functions in data analysis. Technometrics 16(1):1–11.zbMATHMathSciNetCrossRefGoogle Scholar
  73. Wood, S. N. 2006. Generalized Additive Models: An Introduction with R. London: Chapman and Hall.Google Scholar
  74. Yanai, H., K. Takeuchi, and Y. Takane 2011. Projection Matrices, Generalized Inverse Matrices, and Singular Value Decomposition. New York, USA: Springer.zbMATHCrossRefGoogle Scholar
  75. Yee, T. W. and N. D. Mitchell 1991. Generalized additive models in plant ecology. Journal of Vegetation Science 2(5):587–602.CrossRefGoogle Scholar
  76. Zhang, C. 2003. Calibrating the degrees of freedom for automatic data smoothing and effective curve checking. Journal of the American Statistical Association 98(463):609–628.zbMATHMathSciNetCrossRefGoogle Scholar
  77. Zuur, A. F. 2012. A Beginner’s Guide to Generalized Additive Models with R. Newburgh, UK: Highland Statistics Ltd.Google Scholar

Copyright information

© Thomas Yee 2015

Authors and Affiliations

  • Thomas W. Yee
    • 1
  1. 1.Department of StatisticsUniversity of AucklandAucklandNew Zealand

Personalised recommendations