Finite Mixtures of Generalized Linear Regression Models

  • Bettina Grün
  • Friedrich Leisch

Finite mixture models have now been used for more than hundred years (Newcomb (1886), Pearson (1894)). They are a very popular statistical modeling technique given that they constitute a flexible and-easily extensible model class for (1) approximating general distribution functions in a semi-parametric way and (2) accounting for unobserved heterogeneity. The number of applications has tremendously increased in the last decades as model estimation in a frequentist as well as a Bayesian framework has become feasible with the nowadays easily available computing power.

The simplest finite mixture models are finite mixtures of distributions which are used for model-based clustering. In this case the model is given by a convex combination of a finite number of different distributions where each of the distributions is referred to as component. More complicated mixtures have been developed by inserting different kinds of models for each component. An obvious extension is to estimate a generalized linear model (McCullagh and Nelder (1989)) for each component. Finite mixtures of GLMs allow to relax the assumption that the regression coefficients and dispersion parameters are the same for all observations. In contrast to mixed effects models, where it is assumed that the distribution of the parameters over the observations is known, finite mixture models do not require to specify this distribution a-priori but allow to approximate it in a data-driven way.

In a regression setting unobserved heterogeneity for example occurs if important covariates have been omitted in the data collection and hence their influence is not accounted for in the data analysis. In addition in some areas of application the modeling aim is to find groups of observations with similar regression coefficients. In market segmentation (Wedel and Kamakura (2001)) one kind of application among others of finite mixtures of GLMs aims for example at determining groups of consumers with similar price elasticities in order to develop an optimal pricing policy for a market segment.


Market Share Mixture Model Finite Mixture Multinomial Logit Model Finite Mixture Model 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aitkin M (1996) A general maximum likelihood analysis of overdisper-sion in generalized linear models. Statistics and Computing 6:251-262CrossRefGoogle Scholar
  2. Aitkin M (1999) Meta-analysis by random effect modelling in generalized linear models. Statistics in Medicine 18(17-18):2343-2351CrossRefGoogle Scholar
  3. Böhning D, Dietz E, Schlattmann P, Mendonça L, Kirchner U (1999) The zero-inflated Poisson model and the decayed, missing and filled teeth index in dental epidemiology. Journal of the Royal Statistical Society A 162(2):195-209CrossRefGoogle Scholar
  4. Boiteau G, Singh M, Singh RP, Tai GCC, Turner TR (1998) Rate of spread of pvy-n by alate myzus persicae (sulzer) from infected to healthy plants under laboratory conditions. Potato Research 41 (4):335-344CrossRefGoogle Scholar
  5. Celeux G, Diebolt J (1988) A random imputation principle: The stochastic EM algorithm. Rapports de Recherche 901, INRIAGoogle Scholar
  6. Dasgupta A, Raftery AE (1998) Detecting features in spatial point processes with clutter via model-based clustering. Journal of the American Statistical Association 93(441):294-302MATHCrossRefGoogle Scholar
  7. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM-algorithm. Journal of the Royal Statistical Society B 39:1-38MATHMathSciNetGoogle Scholar
  8. Follmann DA, Lambert D (1989) Generalizing logistic regression by non-parametric mixing. Journal of the American Statistical Association 84(405):295-300CrossRefGoogle Scholar
  9. Frühwirth-Schnatter S (2006) Finite Mixture and Markov Switching Models. Springer Series in Statistics, Springer, New YorkGoogle Scholar
  10. Grün B (2006) Identification and estimation of finite mixture models. PhD thesis, Institut für Statistik und Wahrscheinlichkeitstheorie, Technische Universität Wien, Friedrich Leisch, advisorGoogle Scholar
  11. Grün B, Leisch F (2004) Bootstrapping finite mixture models. In: Antoch J (ed) Compstat 2004 — Proceedings in Computational Statistics, Physica Verlag, Heidelberg, pp 1115-1122Google Scholar
  12. Grün B, Leisch F (2006) Fitting finite mixtures of linear regression models with varying & fixed effects in R. In: Rizzi A, Vichi M (eds) Compstat 2006—Proceedings in Computational Statistics, Physica Verlag, Heidelberg, Germany, pp 853-860Google Scholar
  13. Grün B, Leisch F (2007) Flexmix 2.0: Finite mixtures with concomitant variables and varying and fixed effects. Submitted for publicationGoogle Scholar
  14. Grün B, Leisch F (2007) Identifiability of finite mixtures of multinomial logit models with varying and fixed effects, unpublished manuscriptGoogle Scholar
  15. Grün B, Leisch F (2007) Testing for genuine multimodality in finite mixture models: Application to linear regression models. In: Decker R, Lenz HJ (eds) Advances in Data Analysis, Proceedings of the 30th Annual Conference of the Gesellschaft für Klassifikation, SpringerVerlag, Studies in Classification, Data Analysis, and Knowledge Organization, vol 33, pp 209-216Google Scholar
  16. Hennig C (2000) Identifiability of models for clusterwise linear regression. Journal of Classification 17(2):273-296MATHCrossRefMathSciNetGoogle Scholar
  17. Jedidi K, Krider RE, Weinberg CB (1998) Clustering at the movies. Marketing Letters 9(4):393-405CrossRefGoogle Scholar
  18. Krider RE, Li T, Liu Y, Weinberg CB (2005) The lead-lag puzzle of demand and distribution: A graphical method applied to movies. Marketing Science 24(4):635-645CrossRefGoogle Scholar
  19. Leisch F (2004a) Exploring the structure of mixture model components. In: Antoch J (ed) Compstat 2004 — Proceedings in Computational Statistics, Physica Verlag, Heidelberg, pp 1405-1412Google Scholar
  20. Leisch F (2004b) FlexMix: A general framework for finite mixture mod-els and latent class regression in R. Journal of Statistical Software 11 (8), URL
  21. Lindsay BG (1989) Moment matrices: Applications in mixtures. The Annals of Statistics 17(2):722-740MATHCrossRefMathSciNetGoogle Scholar
  22. McCullagh P, Nelder JA (1989) Generalized Linear Models (2nd edition). Chapman and HallGoogle Scholar
  23. McLachlan GJ, Krishnan T (1997) The EM Algorithm and Extensions, 1st edn. John Wiley and SonsGoogle Scholar
  24. Naik PA, Shi P, Tsai CL (2007) Extending the Akaike information criterion to mixture regression models. Journal of the American Statistical Association 102(477):244-254MATHCrossRefMathSciNetGoogle Scholar
  25. Newcomb S (1886) A generalized theory of the combination of observations so as to obtain the best result. American Journal of Mathematics 8:343-366CrossRefMathSciNetGoogle Scholar
  26. Pearson K (1894) Contributions to the mathematical theory of evolu-tion. Philosophical Transactions of the Royal Society A 185:71-110CrossRefGoogle Scholar
  27. R Development Core Team (2007) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, URL
  28. Redner RA, Walker HF (1984) Mixture densities, maximum likelihood and the EM algorithm. SIAM Review 26(2):195-239MATHCrossRefMathSciNetGoogle Scholar
  29. Titterington DM, Smith AFM, Makov UE (1985) Statistical Analysis of Finite Mixture Distributions. WileyGoogle Scholar
  30. Wang P, Puterman ML (1998) Mixed logistic regression models. Journal of Agricultural, Biological, and Environmental Statistics 3 (2):175-200CrossRefMathSciNetGoogle Scholar
  31. Wang P, Puterman ML, Cockburn IM, Le ND (1996) Mixed Poisson regression models with covariate dependent rates. Biometrics 52:381-400MATHCrossRefGoogle Scholar
  32. Wedel M, Kamakura WA (2001) Market Segmentation — Conceptual and Methodological Foundations (2nd edition). Kluwer Academic PublishersGoogle Scholar

Copyright information

© Physica-Verlag Heidelberg 2008

Authors and Affiliations

  • Bettina Grün
    • 1
  • Friedrich Leisch
    • 2
  1. 1.Institut für Statistik und WahrscheinlichkeitstheorieTechnische Universität WienViennaAustria
  2. 2.Department of StatisticsUniversity of MunichMunichGermany

Personalised recommendations