A General Transfer Learning-based Gaussian Mixture Model for Clustering

Abstract

Gaussian mixture model (GMM) is a well-known model-based approach for data clustering. However, when the data samples are insufficient, the classical GMM-based clustering algorithms are not effective anymore. Referring to the idea of transfer clustering methods, this paper proposes a general transfer GMM-based clustering framework, which employs the important knowledge extracted from some known source domain to guide and improve the clustering on the target domain with small-scale data. Specifically, three traditional GMM-based clustering approaches are extended to the corresponding transfer clustering versions. Furthermore, to avoid the negative transfer problem, maximum mean discrepancy (MMD) is introduced to search the most matched source domain to provide more positive guidance for data clustering on the target domain. Experiments on synthetic and real-world datasets demonstrate the efficiency of the presented framework compared with several existing transfer clustering algorithms.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    Andrews, J.L.: Addressing overfitting and underfitting in gaussian model-based clustering. Comput. Stat. Data Anal. 127, 160–171 (2018)

    MathSciNet  Article  Google Scholar 

  2. 2.

    Baktashmotlagh, M., Harandi, M., Lovell, B., Salzmann, M.: Unsupervised domain adaptation by domain invariant projection. In: 2013 IEEE International Conference on Computer Vision, pp. 769–776 (2013)

  3. 3.

    Chatzis, S.: A method for training finite mixture models under a fuzzy clustering principle. Fuzzy Sets Syst. 161(23), 3000–3013 (2010)

    MathSciNet  Article  Google Scholar 

  4. 4.

    Lai, C.Y., Yang, M.S.: Entropy-type classification maximum likelihood algorithms for mixture models. Soft Comput. 15(2), 373–381 (2011)

    Article  Google Scholar 

  5. 5.

    Dai, W., Yang, Q., Xue, G.R., Yu, Y.: Self-taught clustering. In: Proceedings of the 25th International Conference on Machine Learning, pp. 200–207 (2008)

  6. 6.

    Dang, B., Zhou, J., Wang, R., Wang, L., Han, S., Chen, Y.: Transfer learning based kernel fuzzy clustering. In: 2019 International Conference on Fuzzy Theory and Its Applications (iFUZZY), pp. 21–25 (2019)

  7. 7.

    Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. 39, 1–22 (1977)

    MathSciNet  MATH  Google Scholar 

  8. 8.

    Deng, Z., Jiang, Y., Chung, F.L., Choi, K.S., Wang, S.: Transfer prototype-based fuzzy clustering. IEEE Trans. Fuzzy Syst. 24(5), 1210–1232 (2014)

    Article  Google Scholar 

  9. 9.

    Du, M., Ding, S., Xue, Y., Shi, Z.: A novel density peaks clustering with sensitivity of local density and density-adaptive metric. Knowl. Inf. Syst. 59, 1–25 (2018)

    Google Scholar 

  10. 10.

    Gretton, A., Borgwardt, K.M., Rasch, M., Schölkopf, B., Smola, A.J.: A kernel two-sample test. J. Mach. Learn. Res. 13(25), 723–773 (2012)

    MathSciNet  MATH  Google Scholar 

  11. 11.

    Gupta, M., Sinha, A.: Recursive density-based hierarchical clustering in gaussian distributed sensor network. Int. J. Syst. Assurance Eng. Manag. (2020)

  12. 12.

    Hang, W., Chung, F.I., Wang, S.: Transfer affinity propagation-based clustering. Inf. Sci. 348, 337–356 (2016)

    MathSciNet  Article  Google Scholar 

  13. 13.

    He, X., Cai, D., Shao, Y., Bao, H., Han, J.: Laplacian regularized gaussian mixture model for data clustering. IEEE Trans. Knowl. Data Eng. 23(9), 1406–1418 (2011)

    Article  Google Scholar 

  14. 14.

    Iqbal, S., Zhang, C.: A new hesitant fuzzy-based forecasting method integrated with clustering and modified smoothing approach. Int. J. Fuzzy Syst. 22(3), 1104–1117 (2020)

    Article  Google Scholar 

  15. 15.

    Jiang, W., Chung, F.L.: Transfer spectral clustering. In: European Conference on Machine Learning & Knowledge Discovery in Databases, pp. 789–803 (2012)

  16. 16.

    Jiang, W., Liu, W., Chung, Fl: Knowledge transfer for spectral clustering. Pattern Recognit. 81, 484–496 (2018)

    Article  Google Scholar 

  17. 17.

    Ju, Z., Liu, H.: Fuzzy gaussian mixture models. Pattern Recognit. 45(3), 1146–1158 (2012)

    Article  Google Scholar 

  18. 18.

    Kannan, S.: Intelligent object recognition in underwater images using evolutionary-based gaussian mixture model and shape matching. Signal Image Video Process. 1–9 (2020)

  19. 19.

    Long, M., Wang, J., Ding, G., Sun, J., Yu, P.: Transfer feature learning with joint distribution adaptation. In: Proceedings of the 2013 IEEE International Conference on Computer Vision, pp. 2200–2207 (2013)

  20. 20.

    Ma, J., Wang, T.: Entropy penalized automated model selection on gaussian mixture. Int. J. Pattern Recognit. Artif. Intell. 18(8), 1501–1512 (2004)

    Article  Google Scholar 

  21. 21.

    McLachlan, G.J., Basford, K.E.: Mixture models: inference and applications to clustering. Inference Appl. Clust. 38(2) (1988)

  22. 22.

    Memon, K.H., Memon, S., Qureshi, M.A., Alvi, M.B., Kumar, D., Shah, R.A.: Kernel possibilistic fuzzy c-means clustering with local information for image segmentation. Int. J. Fuzzy Syst. 21(1), 321–332 (2019)

    Article  Google Scholar 

  23. 23.

    NENE, S.A.: Columbia object image library(coil-20). Technical Report 5 (1996)

  24. 24.

    Qian, P., Jiang, Y., Deng, Z., Hu, L., Sun, S., Wang, S., Muzic, R.: Cluster prototypes and fuzzy memberships jointly leveraged cross-domain maximum entropy clustering. IEEE Trans. Cybern. 46(1), 181–193 (2016)

    Article  Google Scholar 

  25. 25.

    Quost, B., Denœux, T.: Clustering and classification of fuzzy data using the fuzzy EM algorithm. Fuzzy Sets Syst. 286, 134–156 (2016)

    MathSciNet  Article  Google Scholar 

  26. 26.

    Reddy, C.K., Chiang, H.D., Rajaratnam, B.: Trust-tech-based expectation maximization for learning finite mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 30(7), 1146–1157 (2008)

    Article  Google Scholar 

  27. 27.

    Saranya, S., Poonguzhali, S., Karunakaran, S.: Gaussian mixture model based clustering of manual muscle testing grades using surface electromyogram signals. Phys. Eng. Sci. Med. (2020)

  28. 28.

    Sevillano, X., Socoró, J.C., Alías, F.: Parallel hierarchical architectures for efficient consensus clustering on big multimedia cluster ensembles. Inf. Sci. 511, 212–228 (2020)

    MathSciNet  Article  Google Scholar 

  29. 29.

    Sharma, R., Verma, K.: Fuzzy shared nearest neighbor clustering. Int. J. Fuzzy Syst. 21(6), 2667–2678 (2019)

    Article  Google Scholar 

  30. 30.

    Tran, D., Wagner, M.: Fuzzy entropy clustering. In: Ninth IEEE International Conference on Fuzzy Systems. FUZZ-IEEE 2000 (Cat. No.00CH37063), vol. 1, pp. 152–157 (2000)

  31. 31.

    Wang, Y., Dong, J., Zhou, J., Xu, G., Chen, Y.: Random feature map-based multiple kernel fuzzy clustering with all feature weights. Int. J. Fuzzy Syst. 21(7), 2132–2146 (2019)

    MathSciNet  Article  Google Scholar 

  32. 32.

    Wolfe, J.: Object Cluster Analysis of Social Areas. University of California, Berkeley (1963)

    Google Scholar 

  33. 33.

    Xu, G., Zhou, J., Dong, J., Zhang, T., Chen, L., Han, S., Wang, L., Chen, Y.: Multivariate morphological reconstruction based fuzzy clustering with a weighting multi-channel guided image filter for color image segmentation. Int. J. Mach. Learn. Cybern. (2020). https://doi.org/10.1007/s13042-020-01151-1

    Article  Google Scholar 

  34. 34.

    Yang, M.S., Chang-Chien, S.J., Nataliani, Y.: Unsupervised fuzzy model-based gaussian clustering. Inf. Sci. 481, 1–23 (2019)

    MathSciNet  Article  Google Scholar 

  35. 35.

    Yang, Z., Shrivastava, A.K., Leung, T.K.: Regularized gaussian mixture model for high-dimensional clustering. IEEE Trans. Cybern. 49(10), 3677–3688 (2019)

    Article  Google Scholar 

  36. 36.

    Yeganegi, H., Salami, P., Daliri, mr: A template-based sequential algorithm for online clustering of spikes in extracellular recordings. Cogn. Comput. 12(2), 542–552 (2020)

    Article  Google Scholar 

  37. 37.

    Yu, L., Dang, Y., Yang, G.: Transfer clustering via constraints generated from topics. In: 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3203–3208 (2012)

  38. 38.

    Zhao, X., Li, Y., Zhao, Q.: A fuzzy clustering approach for complex color image segmentation based on gaussian model with interactions between color planes and mixture gaussian model. Int. J. Fuzzy Syst. 20(1), 309–317 (2018)

    MathSciNet  Article  Google Scholar 

  39. 39.

    Zhou, J., Chen, L., Chen, C.L.P., Wang, Y., Li, H.: Uncertain data clustering in distributed peer-to-peer networks. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2392–2406 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grants with No. 61873324, No. 61903156, and No. 61872419, the Natural Science Foundation of Shandong Province under Grant with No. ZR2019MF040 and No. ZR2018LF005.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jin Zhou.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wang, R., Zhou, J., Jiang, H. et al. A General Transfer Learning-based Gaussian Mixture Model for Clustering. Int. J. Fuzzy Syst. (2021). https://doi.org/10.1007/s40815-020-01016-3

Download citation

Keywords

  • Gaussian mixture model
  • Transfer clustering
  • Maximum mean discrepancy