Penalized Splines, Mixed Models and Bayesian Ideas



The paper describes the link between penalized spline smoothing and Linear Mixed Models and how these two models form a practical and theoretically interesting partnership. As offspring of this partnership one can not only estimate the smoothing parameter in a Maximum Likelihood framework but also utilize the Mixed Model technology to derive numerically handy solutions to more general questions and problems. Two particular examples are discussed in this paper. The first contribution demonstrates penalized splines and Linear Mixed Models in a classification context. Secondly, an even broader framework is pursued, mirroring the Bayesian paradigm combined with simple approximate numerical solutions for model selection.


Bayesian Information Criterion Bayesian Idea Generalize Additive Model Marginal Likelihood Laplace Approximation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Breslow, N. E. & Clayton, D. G. (1993). Approximate inference in generalized linear mixed model. Journal of the American Statistical Association 88: 9–25.MATHCrossRefGoogle Scholar
  2. Claeskens, G., Krivobokova, T. & Opsomer, J. (2009). Asymptotic properties of penalized spline estimators. Biometrika, to appear.Google Scholar
  3. Crainiceanu, M. & Ruppert, D. (2004). Likelihood ratio tests in linear mixed models with one variance component. Journal of the Royal Statistical Society B 66: 165–185.MATHCrossRefMathSciNetGoogle Scholar
  4. Crainiceanu, C., Ruppert, D., Claeskens, G. & Wand, M.P. (2005). Likelihood ratio tests of polynomial regression against a general nonparametric alternative. Biometrika 92: 91–103.MATHCrossRefMathSciNetGoogle Scholar
  5. de Boor, C. (1972). On calculating with B-splines. Journal of Approximation Theory 6: 50–62.MATHCrossRefMathSciNetGoogle Scholar
  6. Eilers, P.H.C. & Marx, B.D. (1996). Flexible smoothing with B-splines and penalties. Statistical Science 11: 89–121.MATHCrossRefMathSciNetGoogle Scholar
  7. Hall, P. & Opsomer, J. (2005). Theory for penalised spline regression. Biometrika 92: 105–118.MATHCrossRefMathSciNetGoogle Scholar
  8. Hastie, T. & Tibshirani, R. (1990). Generalized Additive Models. London: Chapman & Hall.MATHGoogle Scholar
  9. Hastie, T., Tibshirani, R. & Friedman, J. (2001). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York.MATHGoogle Scholar
  10. Kauermann, G., Krivobokova, T. & Fahrmeir, L. (2009). Some asymptotic results on generalized penalized spline smoothing. Journal of the Royal Statistical Society, Series B 71: 487–503.CrossRefGoogle Scholar
  11. Kauermann, G. & Opsomer, J. (2009). Data-driven selection of the spline dimension in penalized spline regression. Technical Report. Google Scholar
  12. Kauermann, G., Ormerod, J. & Wand, M. (2009). Parsimonious classification via generalized linear mixed models. Journal of Classification, to appear.Google Scholar
  13. Krivobokova, T. & Kauermann, G. (2007). A note on penalized spline smoothing with correlated errors. Journal of the American Statistical Association 102: 1328–1337.MATHCrossRefMathSciNetGoogle Scholar
  14. Li, Y. & Ruppert, D. (2008). On the asymptotics of penalized splines. Biometrika, to appear.Google Scholar
  15. Ngo, L. & Wand, M. P. (2004). Smoothing with mixed model software. Journal of Statistical Software 9: 1–54.Google Scholar
  16. O’Sullivan, F. (1986). A statistical perspective on ill-posed inverse problems (c/r: P519-527). Statistical Science 1: 502–518.MATHCrossRefMathSciNetGoogle Scholar
  17. Rue, H., Martino, S. & Chopin, N. (2009). Approximate Bayesian inference for latent gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society: Series B 71: 319–392.CrossRefGoogle Scholar
  18. Ruppert, D. (2002). Selecting the number of knots for penalized splines. Journal of Computational and Graphical Statistics 11: 735–757.CrossRefMathSciNetGoogle Scholar
  19. Ruppert, R., Wand, M. & Carroll, R. (2003). Semiparametric Regression. Cambridge: Cambridge University Press.MATHGoogle Scholar
  20. Ruppert, D., Wand, M. & Carroll, R. (2009). Semiparametric regression during 2003–2007. Technical Report. Google Scholar
  21. Searle, S., Casella, G. & McCulloch, C. (1992). Variance Components. Wiley, New York.MATHCrossRefGoogle Scholar
  22. Tutz, G. & Binder, H. (2006). Generalized additive modelling with implicit variable selection by likelihood based boosting. Biometrics 62: 961–971.MATHCrossRefMathSciNetGoogle Scholar
  23. Vaida, F. & Blanchard, S. (2005). Conditional akaike information for mixed effects models. Biometrika 92: 351–370.MATHCrossRefMathSciNetGoogle Scholar
  24. Wager, C., Vaida, F. & Kauermann, G. (2007). Model selection for P-spline smoothing using Akaike information criteria. Australian and New Zealand Journal of Statistics 49: 173–190.MATHCrossRefMathSciNetGoogle Scholar
  25. Wand, M. (2003). Smoothing and mixed models. Computational Statistics 18: 223–249.MATHGoogle Scholar
  26. Wand, M. & Ormerod, J. (2008). On semiparametric regression with O’Sullivan penalized splines. Australian and New Zealand Journal of Statistics 50: 179–198.MATHCrossRefMathSciNetGoogle Scholar
  27. Wood, S. (2006). Generalized Additive Models. London: Chapman & Hall.MATHGoogle Scholar
  28. Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti, (eds. P. K. Goel & A. Zellner), pp. 233–243. North-Holland/Elsevier.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  1. 1.Dep of Business Administration and EconomicsCentre for Statistics, Bielefeld UniversityBielefeldGermany

Personalised recommendations