Mixtures of skewed matrix variate bilinear factor analyzers

Abstract

In recent years, data have become increasingly higher dimensional and, therefore, an increased need has arisen for dimension reduction techniques for clustering. Although such techniques are firmly established in the literature for multivariate data, there is a relative paucity in the area of matrix variate, or three-way, data. Furthermore, the few methods that are available all assume matrix variate normality, which is not always sensible if cluster skewness or excess kurtosis is present. Mixtures of bilinear factor analyzers using skewed matrix variate distributions are proposed. In all, four such mixture models are presented, based on matrix variate skew-t, generalized hyperbolic, variance-gamma, and normal inverse Gaussian distributions, respectively.

This is a preview of subscription content, log in to check access.

Fig. 1

References

  1. Anderlucci L, Viroli C (2015) Covariance pattern mixture models for the analysis of multivariate heterogeneous longitudinal data. Ann Appl Stat 9(2):777–800

    MathSciNet  MATH  Google Scholar 

  2. Andrews JL, McNicholas PD (2011) Extending mixtures of multivariate t-factor analyzers. Stat Comput 21(3):361–373

    MathSciNet  MATH  Google Scholar 

  3. Andrews JL, McNicholas PD (2012) Model-based clustering, classification, and discriminant analysis via mixtures of multivariate \(t\)-distributions: the \(t\)EIGEN family. Stat Comput 22(5):1021–1029

    MathSciNet  MATH  Google Scholar 

  4. Baum LE, Petrie T, Soules G, Weiss N (1970) A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann Math Stat 41:164–171

    MathSciNet  MATH  Google Scholar 

  5. Bezanson J, Edelman A, Karpinski S, Shah V B (2017) Julia: a fresh approach to numerical computing. SIAM Rev 59(1):65–98

    MathSciNet  MATH  Google Scholar 

  6. Browne RP, McNicholas PD (2015) A mixture of generalized hyperbolic distributions. Can J Stat 43(2):176–198

    MathSciNet  MATH  Google Scholar 

  7. Chen JT, Gupta AK (2005) Matrix variate skew normal distributions. Statistics 39(3):247–253

    MathSciNet  MATH  Google Scholar 

  8. Dang UJ, Browne RP, McNicholas PD (2015) Mixtures of multivariate power exponential distributions. Biometrics 71(4):1081–1089

    MathSciNet  MATH  Google Scholar 

  9. Domínguez-Molina JA, González-Farías G, Ramos-Quiroga R, Gupta AK (2007) A matrix variate closed skew-normal distribution with applications to stochastic frontier analysis. Commun Stat Theory Methods 36(9):1691–1703

    MathSciNet  MATH  Google Scholar 

  10. Franczak BC, Browne RP, McNicholas PD (2014) Mixtures of shifted asymmetric Laplace distributions. IEEE Trans Pattern Anal Mach Intell 36(6):1149–1157

    Google Scholar 

  11. Gallaugher MPB, McNicholas PD (2017) A matrix variate skew-t distribution. Stat 6(1):160–170

    MathSciNet  Google Scholar 

  12. Gallaugher MPB, McNicholas PD (2018a) Finite mixtures of skewed matrix variate distributions. Pattern Recogn 80:83–93

    Google Scholar 

  13. Gallaugher MPB, McNicholas PD (2018b) Mixtures of matrix variate bilinear factor analyzers. In: Proceedings of the joint statistical meetings. American Statistical Association, Alexandria, VA. arXiv:1712.08664

  14. Gallaugher MPB, McNicholas PD (2019) Three skewed matrix variate distributions. Stat Probab Lett 145:103–109

    MathSciNet  MATH  Google Scholar 

  15. Ghahramani Z, Hinton GE (1997) The EM algorithm for factor analyzers. Technical report CRG-TR-96-1, University of Toronto, Toronto, Canada

  16. Harrar SW, Gupta AK (2008) On matrix variate skew-normal distributions. Statistics 42(2):179–194

    MathSciNet  MATH  Google Scholar 

  17. Karlis D, Santourian A (2009) Model-based clustering with non-elliptically contoured distributions. Stat Comput 19(1):73–83

    MathSciNet  Google Scholar 

  18. Lee S, McLachlan GJ (2014) Finite mixtures of multivariate skew t-distributions: some recent and new results. Stat Comput 24:181–202

    MathSciNet  MATH  Google Scholar 

  19. Lin T-I (2010) Robust mixture modeling using multivariate skew t distributions. Stat Comput 20(3):343–356

    MathSciNet  Google Scholar 

  20. Lin T-I, McNicholas PD, Hsiu JH (2014) Capturing patterns via parsimonious t mixture models. Stat Probab Lett 88:80–87

    MathSciNet  MATH  Google Scholar 

  21. McNicholas PD (2010) Model-based classification using latent Gaussian mixture models. J Stat Plan Inference 140(5):1175–1181

    MathSciNet  MATH  Google Scholar 

  22. McNicholas PD (2016) Mixture model-based classification. Chapman & Hall/CRC Press, Boca Raton

    Google Scholar 

  23. McNicholas PD, Murphy TB (2008) Parsimonious Gaussian mixture models. Stat Comput 18(3):285–296

    MathSciNet  Google Scholar 

  24. McNicholas PD, Murphy TB (2010) Model-based clustering of microarray expression data via latent Gaussian mixture models. Bioinformatics 26(21):2705–2712

    Google Scholar 

  25. McNicholas PD, Tait PA (2019) Data science with Julia. Chapman & Hall/CRC Press, Boca Raton

    Google Scholar 

  26. McNicholas SM, McNicholas PD, Browne RP (2017) A mixture of variance-gamma factor analyzers. In: Ahmed SE (ed) Big and complex data analysis: methodologies and applications. Springer, Cham, pp 369–385

    Google Scholar 

  27. Melnykov V, Zhu X (2018) On model-based clustering of skewed matrix data. J Multivar Anal 167:181–194

    MathSciNet  MATH  Google Scholar 

  28. Melnykov V, Zhu X (2019) Studying crime trends in the USA over the years 2000–2012. Adv Data Anal Classif 13(1):325–341

    MathSciNet  MATH  Google Scholar 

  29. Meng X-L, van Dyk D (1997) The EM algorithm—an old folk song sung to a fast new tune (with discussion). J R Stat Soc B 59(3):511–567

    MATH  Google Scholar 

  30. Morris K, McNicholas PD (2013) Dimension reduction for model-based clustering via mixtures of shifted asymmetric Laplace distributions. Stat Probab Lett 83(9):2088–2093

    MathSciNet  MATH  Google Scholar 

  31. Murray PM, Browne RB, McNicholas PD (2014a) Mixtures of skew-t factor analyzers. Comput Stat Data Anal 77:326–335

    MathSciNet  MATH  Google Scholar 

  32. Murray PM, McNicholas PD, Browne RB (2014b) A mixture of common skew-\(t\) factor analyzers. Stat 3(1):68–82

    MathSciNet  MATH  Google Scholar 

  33. Murray PM, Browne RB, McNicholas PD (2017) Hidden truncation hyperbolic distributions, finite mixtures thereof, and their application for clustering. J Multivar Anal 161:141–156

    MathSciNet  MATH  Google Scholar 

  34. Peel D, McLachlan GJ (2000) Robust mixture modelling using the t distribution. Stat Comput 10(4):339–348

    Google Scholar 

  35. Počuča N, Gallaugher MPB, McNicholas PD (2019) MatrixVariate.jl: a complete statistical framework for analyzing matrix variate data. Julia package version 0.2.0. http://github.com/nikpocuca/MatrixVariate.jl

  36. Scott AJ, Symons MJ (1971) Clustering methods based on likelihood ratio criteria. Biometrics 27:387–397

    Google Scholar 

  37. Tait PA, McNicholas PD (2019) Clustering higher order data: finite mixtures of multidimensional arrays. arXiv preprint arXiv:1907.08566

  38. Tang Y, Browne RP, McNicholas PD (2018) Flexible clustering of high-dimensional data via mixtures of joint generalized hyperbolic distributions. Stat 7(1):e177

    MathSciNet  Google Scholar 

  39. Tiedeman DV (1955) On the study of types. In: Sells SB (ed) Symposium on pattern analysis. Air University, U.S.A.F. School of Aviation Medicine, Randolph Field

    Google Scholar 

  40. Tortora C, Franczak BC, Browne RP, McNicholas PD (2019) A mixture of coalesced generalized hyperbolic distributions. J Classif 36(1):26–57

    MathSciNet  MATH  Google Scholar 

  41. Viroli C (2011) Finite mixtures of matrix normal distributions for classifying three-way data. Stat Comput 21(4):511–522

    MathSciNet  MATH  Google Scholar 

  42. Viroli C (2011) Model based clustering for three-way data structures. Bayesian Anal 6:573–602

    MathSciNet  MATH  Google Scholar 

  43. Vrbik I, McNicholas PD (2012) Analytic calculations for the EM algorithm for multivariate skew-t mixture models. Stat Probab Lett 82(6):1169–1174

    MathSciNet  MATH  Google Scholar 

  44. Vrbik I, McNicholas PD (2014) Parsimonious skew mixture models for model-based clustering and classification. Comput Stat Data Anal 71:196–210

    MathSciNet  MATH  Google Scholar 

  45. Wishart J (1928) The generalised product moment distribution in samples from a normal multivariate population. Biometrika 20A(1/2):32–52

    MATH  Google Scholar 

  46. Wolfe JH (1965) A computer program for the maximum likelihood analysis of types. Technical bulletin 65-15, U.S. Naval Personnel Research Activity

  47. Xie X, Yan S, Kwok JT, Huang TS (2008) Matrix-variate factor analysis and its applications. IEEE Trans Neural Netw 19(10):1821–1826

    Google Scholar 

  48. Yu S, Bi J, Ye J (2008) Probabilistic interpretations and extensions for a family of 2D PCA-style algorithms. In: Workshop data mining using matrices and tensors (DMMT 08): proceedings of a workshop held in conjunction with the 14th ACM SIGKDD international conference on knowledge discovery and data mining (SIGKDD 2008)

  49. Zhao J, Philip L, Kwok JT (2012) Bilinear probabilistic principal component analysis. IEEE Trans Neural Netw Learn Syst 23(3):492–503

    Google Scholar 

Download references

Acknowledgements

The authors are grateful for the helpful comments of two anonymous reviewers. This work was supported by a Vanier Canada Graduate Scholarship (Gallaugher), the Canada Research Chairs program (McNicholas), and an E.W.R. Steacie Memorial Fellowship (McNicholas).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Paul D. McNicholas.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 146 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gallaugher, M.P.B., McNicholas, P.D. Mixtures of skewed matrix variate bilinear factor analyzers. Adv Data Anal Classif 14, 415–434 (2020). https://doi.org/10.1007/s11634-019-00377-4

Download citation

Keywords

  • Clustering
  • Factor analysis
  • Kurtosis
  • Skewed
  • Matrix variate distribution
  • Mixture models

Mathematics Subject Classification

  • 62H30
  • 62H25