# Fitting monotone polynomials in mixed effects models

- 247 Downloads

## Abstract

We provide a method for fitting monotone polynomials to data with both fixed and random effects. In pursuit of such a method, a novel approach to least squares regression is proposed for models with functional constraints. The new method is able to fit models with constrained parameter spaces that are closed and convex, and is used in conjunction with an expectation–maximisation algorithm to fit monotone polynomials with mixed effects. The resulting mixed effects models have constrained mean curves and have the flexibility to include either unconstrained or constrained subject-specific curves. This new methodology is demonstrated on real-world repeated measures data with an application from sleep science. Code to fit the methods described in this paper is available online.

## Keywords

Monotone polynomials Monotone regression Mixed effects Random effects Shape constraints## Supplementary material

## References

- Afriat, S.: Theory of maxima and the method of Lagrange. SIAM J. Appl. Math.
**20**(3), 343–357 (1971)MathSciNetzbMATHGoogle Scholar - Auspitz, R., Lieben, R.: Untersuchungen über die Theorie des Preises. Duncker & Humblot, Berlin (1889)zbMATHGoogle Scholar
- Barlow, R.E., Brunk, H.D.: The isotonic regression problem and its dual. J. Am. Stat. Assoc.
**67**(337), 140–147 (1972)MathSciNetzbMATHGoogle Scholar - Barlow, R.E., Bartholomew, D.J., Bremner, J., Brunk, H.D.: Statistical Inference Under Order Restrictions: The Theory and Application of Isotonic Regression. Wiley, New York (1972)zbMATHGoogle Scholar
- Bates, D., Mächler, M., Bolker, B., Walker, S.: Fitting linear mixed-effects models using lme4. J. Stat. Softw.
**67**(1), 1–48 (2015)Google Scholar - Belenky, G., Wesensten, N.J., Thorne, D.R., Thomas, M.L., Sing, H.C., Redmond, D.P., Russo, M.B., Balkin, T.J.: Patterns of performance degradation and restoration during sleep restriction and subsequent recovery: a sleep dose-response study. J. Sleep Res.
**12**(1), 1–12 (2003)Google Scholar - Booth, J.G., Hobert, J.P.: Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm. J. R. Stat. Soc. Ser. B (Stat. Methodol.)
**61**(1), 265–285 (1999)zbMATHGoogle Scholar - Bradley, R.A., Srivastava, S.S.: Correlation in polynomial regression. Am. Stat.
**33**(1), 11–14 (1979)zbMATHGoogle Scholar - Cassioli, A., Lorenzo, D.D., Sciandrone, M.: On the convergence of inexact block coordinate descent methods for constrained optimization. Eur. J. Oper. Res.
**231**(2), 274–281 (2013)MathSciNetzbMATHGoogle Scholar - Chen, J., Zhang, D., Davidian, M.: A Monte Carlo EM algorithm for generalized linear mixed models with flexible random effects distribution. Biostatistics
**3**(3), 347–360 (2002)zbMATHGoogle Scholar - Damien, P., Walker, S.G.: Sampling truncated normal, beta, and gamma densities. J. Comput. Graph. Stat.
**10**(2), 206–215 (2001)MathSciNetGoogle Scholar - De Boor, C.: A Practical Guide to Splines. Springer, New York (1978)zbMATHGoogle Scholar
- Dette, H., Neumeyer, N., Pilz, K.F., et al.: A simple nonparametric estimator of a strictly monotone regression function. Bernoulli
**12**(3), 469–490 (2006)MathSciNetzbMATHGoogle Scholar - Dierckx, I.P.: An algorithm for cubic spline fitting with convexity constraints. Computing
**24**(4), 349–371 (1980)MathSciNetzbMATHGoogle Scholar - Elphinstone, C.D.: A target distribution model for nonparametric density estimation. Commun. Stat. Theory Methods
**12**(2), 161–198 (1983)MathSciNetGoogle Scholar - Emerson, P.L.: Numerical construction of orthogonal polynomials from a general recurrence formula. Biometrics
**24**(3), 695–701 (1968)MathSciNetGoogle Scholar - Forsythe, G.E.: Generation and use of orthogonal polynomials for data-fitting with a digital computer. J. Soc. Ind. Appl. Math.
**5**(2), 74–88 (1957)MathSciNetzbMATHGoogle Scholar - Friedman, J., Tibshirani, R.: The monotone smoothing of scatterplots. Technometrics
**26**(3), 243–250 (1984)Google Scholar - Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika
**57**(1), 97–109 (1970)MathSciNetzbMATHGoogle Scholar - Hawkins, D.M.: Fitting monotonic polynomials to data. Comput. Stat.
**9**(3), 233–247 (1994)zbMATHGoogle Scholar - Hazelton, M.L., Turlach, B.A.: Semiparametric regression with shape-constrained penalized splines. Comput. Stat. Data Anal.
**55**(10), 2871–2879 (2011)MathSciNetzbMATHGoogle Scholar - Hodges, J.S.: Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects. CRC Press, Boca Raton (2013)zbMATHGoogle Scholar
- Holland, P.W., Welsch, R.E.: Robust regression using iteratively reweighted least-squares. Commun. Stat. Theory Methods
**6**(9), 813–827 (1977)zbMATHGoogle Scholar - Hornung, U.: Monotone spline interpolation. In: Collatz, L., Meinardus, G., Werner, H. (eds.) Numerische Methoden der Approximationstheorie, pp. 172–191. Birkhäuser, Basel (1978)Google Scholar
- Horrace, W.C.: Some results on the multivariate truncated normal distribution. J. Multivar. Anal.
**94**(1), 209–221 (2005)MathSciNetzbMATHGoogle Scholar - Kelly, C., Rice, J.: Monotone smoothing with application to dose-response curves and the assessment of synergism. Biometrics
**46**(4), 1071–1085 (1990)Google Scholar - Laird, N., Lange, N., Stram, D.: Maximum likelihood computations with repeated measures: application of the EM algorithm. J. Am. Stat. Assoc.
**82**(397), 97–105 (1987)MathSciNetzbMATHGoogle Scholar - Lee, L.F.: On the first and second moments of the truncated multi-normal distribution and a simple estimator. Econ. Lett.
**3**(2), 165–169 (1979)Google Scholar - Leppard, P., Tallis, G.M.: Algorithm AS 249: evaluation of the mean and covariance of the truncated multinormal distribution. J. R. Stat. Soc. Ser. C (Appl. Stat.)
**38**(3), 543–553 (1989)zbMATHGoogle Scholar - Levine, R.A., Casella, G.: Implementations of the Monte Carlo EM algorithm. J. Comput. Graph. Stat.
**10**(3), 422–439 (2001)MathSciNetGoogle Scholar - Lindstrom, M.J., Bates, D.M.: Newton-Raphson and EM algorithms for linear mixed-effects models for repeated-measures data. J. Am. Stat. Assoc.
**83**(404), 1014–1022 (1988)MathSciNetzbMATHGoogle Scholar - Meng, X.L., Rubin, D.B.: Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika
**80**(2), 267–278 (1993)MathSciNetzbMATHGoogle Scholar - Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys.
**21**(6), 1087–1092 (1953)Google Scholar - Milgrom, P., Segal, I.: Envelope theorems for arbitrary choice sets. Econometrica
**70**(2), 583–601 (2002)MathSciNetzbMATHGoogle Scholar - Murray, K., Müller, S., Turlach, B.A.: Revisiting fitting monotone polynomials to data. Comput. Stat.
**28**(5), 1989–2005 (2013)MathSciNetzbMATHGoogle Scholar - Murray, K., Müller, S., Turlach, B.A.: Fast and flexible methods for monotone polynomial fitting. Stat. Comput. Simul.
**86**(15), 2946–2966 (2016)MathSciNetGoogle Scholar - Narula, S.C.: Orthogonal polynomial regression. Int. Stat. Rev.
**47**(1), 31–36 (1979)zbMATHGoogle Scholar - Pinheiro, J.C., Bates, D.M.: Unconstrained parametrizations for variance-covariance matrices. Stat. Comput.
**6**(3), 289–296 (1996)Google Scholar - R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.r-project.org (2016)
- Ramsay, J.O.: Estimating smooth monotone functions. J. R. Stat. Soc. Ser. B (Stat. Methodol.)
**60**(2), 365–375 (1998)MathSciNetzbMATHGoogle Scholar - Samuelson, P.A.: Foundations of Economic Analysis. Harvard University Press, Cambridge (1947)zbMATHGoogle Scholar
- Schmidt, T.: Really pushing the envelope: early use of the envelope theorem by Auspitz and Lieben. Hist. Polit. Econ.
**36**(1), 103–129 (2004)MathSciNetGoogle Scholar - Tallis, G.M.: The moment generating function of the truncated multi-normal distribution. J. R. Stat. Soc. Ser. B (Methodol.)
**23**(1), 223–229 (1961)MathSciNetzbMATHGoogle Scholar - Tuddenham, R.D., Snyder, M.M.: Physical growth of California boys and girls from birth to eighteen years. Publ. Child Dev. Univ. Calif. Berkeley
**1**(2), 183–364 (1954)Google Scholar - Turlach, B.A.: Shape constrained smoothing using smoothing splines. Comput. Stat.
**20**(1), 81–104 (2005)MathSciNetzbMATHGoogle Scholar - Turlach, B.A., Murray, K.: MonoPoly: Functions to Fit Monotone Polynomials. http://cran.r-project.org/package=MonoPoly (2016). R package version 0.3-8
- Utreras, F.I.: Convergence rates for monotone cubic spline interpolation. J. Approx. Theory
**36**(1), 86–90 (1982)MathSciNetzbMATHGoogle Scholar - Utreras, F.I.: Smoothing noisy data under monotonicity constraints existence, characterization and convergence rates. Numer. Math.
**47**(4), 611–625 (1985)MathSciNetzbMATHGoogle Scholar - Wilhelm, S., Manjunath, B.G.: tmvtnorm: Truncated Multivariate Normal and Student t Distribution. http://cran.r-project.org/package=tmvtnorm (2015). R package version 1.4-10
- Wong, Y.: An application of orthogonalization process to the theory of least squares. Ann. Math. Stat.
**6**(2), 53–75 (1935)zbMATHGoogle Scholar - Zadrozny, B., Elkan, C.: Transforming classifier scores into accurate multiclass probability estimates. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 694–699. ACM (2002)Google Scholar
- Zimmerman, D.L., Núñez-Antón, V.: Parametric modelling of growth curve data: an overview. Test
**10**(1), 1–73 (2001)MathSciNetzbMATHGoogle Scholar