Self-learning Procedures for a Kernel Fuzzy Clustering System

  • Zhengbing Hu
  • Yevgeniy Bodyanskiy
  • Oleksii K. Tyshchenko
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 754)

Abstract

The paper exemplifies several self-learning methods through the prism of diverse objective functions used for training a kernel fuzzy clustering system. A self-learning process for synaptic weights is implemented in terms of the competitive learning concept and the probabilistic fuzzy clustering approach. The main feature of the introduced fuzzy clustering system is its capability to cluster data in an online way under conditions when clusters are rather likely to be of an arbitrary shape (which cannot usually be separated in a linear manner) and to be mutually intersecting. Generally speaking, the offered system’s topology is mainly based on both the fuzzy clustering neural network by Kohonen and the general regression neural network. When it comes to training this hybrid system, it is grounded on both the lazy and optimization-based learning concepts.

Keywords

Self-learning procedure Fuzzy clustering Computational intelligence Adaptive neuro-fuzzy system Objective function 

Notes

Acknowledgment

This scientific work was financially supported by self-determined research funds of CCNU from the colleges’ basic research and operation of MOE (CCNU16A02015). The third author also acknowledges the support of the Visegrad Scholarship Program—EaP #51700967 funded by the International Visegrad Fund (IVF).

References

  1. 1.
    Kruse, R., Borgelt, C., Klawonn, F., Moewes, C., Steinbrecher, M., Held, P.: Computational Intelligence. A Methodological Introduction. Springer, Berlin (2013)MATHGoogle Scholar
  2. 2.
    Mumford, C.L., Jain, L.C.: Computational Intelligence. Springer, Berlin (2009)CrossRefGoogle Scholar
  3. 3.
    Kohonen, T.: Self-Organizing Maps. Springer, Berlin (1995)CrossRefGoogle Scholar
  4. 4.
    Haykin, S.: Neural Networks and Learning Machines. Prentice Hall, Upper Saddle River (2009)Google Scholar
  5. 5.
    Delen, D.: Real-World Data Mining: Applied Business Analytics and Decision Making. Pearson FT Press, Upper Saddle River (2015)Google Scholar
  6. 6.
    Aggarwal, C.C.: A Data Mining: The Textbook. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  7. 7.
    Larose, D.T.: Discovering Knowledge in Data: An Introduction to Data Mining. Wiley, Hoboken (2014)CrossRefGoogle Scholar
  8. 8.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Data Mining, Inference, and Prediction. Springer Science & Business Media, LLC, New York (2009)MATHGoogle Scholar
  9. 9.
    Aggarwal, C.C., Reddy, C.K.: Data Clustering: Algorithms and Applications. CRC Press, Boca Raton (2014)MATHGoogle Scholar
  10. 10.
    Gosain, A., Dahiya, S.: Performance analysis of various fuzzy clustering algorithms: a review. Procedia Comput. Sci. 79, 100–111 (2016)CrossRefGoogle Scholar
  11. 11.
    Xu, R., Wunsch, D.C.: Clustering. IEEE Press Series on Computational Intelligence. Wiley, Hoboken (2009)Google Scholar
  12. 12.
    Yang, M.-S., Chang-Chien, S.-J., Hung, W.-L.: An unsupervised clustering algorithm for data on the unit hypersphere. Appl. Soft Comput. 42, 290–313 (2016)CrossRefGoogle Scholar
  13. 13.
    Babichev, S., Taif, M.A., Lytvynenko, V.: Inductive model of data clustering based on the agglomerative hierarchical algorithm. In: The 2016 IEEE First International Conference on Data Stream Mining and Processing (DSMP), pp. 19–22, Lviv (2016)Google Scholar
  14. 14.
    Babichev, S., Lytvynenko, V., Oypenko, V.: Implementation of the objective clustering inductive technology based on DBSCAN clustering algorithm. In: The XII International Scientific and Technical Conference “Computer Sciences and Information Technologies” (CSIT), pp. 479–484, Lviv (2017)Google Scholar
  15. 15.
    Ivanov, Y., Peleshko, D., Makoveychuk, O., Izonin, I., Malets, I., Lotoshunska, N, Batyuk, D.: Adaptive moving object segmentation algorithms in cluttered environments. In: 2015 15th International Conference the Experience of Designing and Application of CAD Systems in Microelectronics (CADSM), pp. 97–99, Lviv (2015)Google Scholar
  16. 16.
    Babichev, S., Lytvynenko, V., Korobchynskyi, M., Taif, M.: Objective clustering inductive technology of gene expression sequences features. Commun. Comput. Inf. Sci. 716, 359–372 (2016)Google Scholar
  17. 17.
    Izonin, I., Tkachenko, R., Peleshko, D., Rak T., Batyuk, D.: Learning-based image super-resolution using weight coefficients of synaptic connections. In: 2015 Xth International Scientific and Technical Conference “Computer Sciences and Information Technologies” (CSIT), pp. 25–29, Lviv (2015)Google Scholar
  18. 18.
    Babichev, S.A., Kornelyuk, A.I., Lytvynenko, V.I., Osypenko, V.: Computational analysis of microarray gene expression profiles of lung cancer. Biopolym. Cell 32(1), 70–79 (2016)CrossRefGoogle Scholar
  19. 19.
    Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York (1981)CrossRefGoogle Scholar
  20. 20.
    Hu, Z., Bodyanskiy, Y.V., Tyshchenko, O.K., Samitova, V.O.: Fuzzy clustering data given on the ordinal scale based on membership and likelihood functions sharing. Int. J. Intell. Syst. Appl. (IJISA) 9(2), 1–9 (2017).  https://doi.org/10.5815/ijisa.2017.02.01CrossRefGoogle Scholar
  21. 21.
    Hu, Z., Bodyanskiy, Y.V., Tyshchenko, O.K., Samitova, V.O.: Fuzzy clustering data given in the ordinal scale. Int. J. Intell. Syst. Appl. (IJISA) 9(1), 67–74 (2017).  https://doi.org/10.5815/ijisa.2017.01.07CrossRefGoogle Scholar
  22. 22.
    Bodyanskiy, Y., Vynokurova, O., Savvo, V., Tverdokhlib, T., Mulesa, P.: Hybrid clustering-classification neural network in the medical diagnostics of the reactive arthritis. Int. J. Intell. Syst. Appl. (IJISA) 8(8), 1–9 (2016).  https://doi.org/10.5815/ijisa.2016.08.01CrossRefGoogle Scholar
  23. 23.
    Hu, Z., Bodyanskiy, Y.V., Tyshchenko, O.K., Tkachov, V.M.: Fuzzy clustering data arrays with omitted observations. Int. J. Intell. Syst. Appl. (IJISA) 9(6), 24–32 (2017).  https://doi.org/10.5815/ijisa.2017.06.03CrossRefGoogle Scholar
  24. 24.
    Kung, S.Y.: Kernel Methods and Machine Learning. University Press, Cambridge (2014)CrossRefGoogle Scholar
  25. 25.
    Czarnowski, I., Jędrzejowicz, P.: Kernel-based fuzzy c-means clustering algorithm for rbf network initialization. In: Czarnowski, I., Caballero, A.M., Howlett, R.J., Jain, L.C. (eds.) Intelligent Decision Technologies 2016: Proceedings of the 8th KES International Conference on Intelligent Decision Technologies (KES-IDT 2016), Part I, pp. 337–347. Springer International Publishing, Cham (2016)CrossRefGoogle Scholar
  26. 26.
    Saikumar, T., Neenu Preetam, I.: Optimized kernel fuzzy c means (OKFCM) clustering algorithm on level set method for noisy images. In: 2013 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), pp. 880–884. IEEE (2013)Google Scholar
  27. 27.
    Zhang, Z., Havens, T.C.: Scalable approximation of kernel fuzzy c-means. In: 2013 IEEE International Conference on Big Data, pp. 161–168. IEEE (2013)Google Scholar
  28. 28.
    Girolami, M.: Mercer kernel based clustering in feature space. IEEE Trans. Neural Netw. 13(3), 780–784 (2002)CrossRefGoogle Scholar
  29. 29.
    Kim, D.-W., Lee, K., Lee, K.H.: Evaluation of the performance of clustering algorithms in kernel-based feature space. Pattern Recogn. 35, 2267–2278 (2002)CrossRefGoogle Scholar
  30. 30.
    MacDonald, D., Fyfe, C.: Clustering in data space and feature space. In: Proceedings of European Symposium on Artificial Neural Networks, ESANN 2002, pp. 137–142, Bruges (2002)Google Scholar
  31. 31.
    Camastra, F., Verri, A.: A novel kernel method for clustering. IEEE Trans. Pattern Anal. Mach. Intell. 5, 801–805 (2005)CrossRefGoogle Scholar
  32. 32.
    Zhang, D.-Q., Chen, S.-C.: Fuzzy clustering using kernel method. In: Proceedings of International Conference on Control and Automation, ICCA 2002, pp. 162–163 (2002)Google Scholar
  33. 33.
    Zhang, D.-Q., Chen, S.-C.: Kernel based fuzzy and possibilistic c-means clustering. In: Proceedings of International Conference on Artificial Neural Networks, ICANN 2003, pp. 122–125 (2003)Google Scholar
  34. 34.
    Miyamoto, S., Mizutani, K.: Fuzzy multiset space and c-means clustering using kernels with application to information retrieval. In: Lecture Notes on Artificial Intelligence, vol. 2715, pp. 387–395. Springer, Heidelberg (2003)Google Scholar
  35. 35.
    Zhang, D.-Q., Chen, S.-C.: A novel kernelized fuzzy c-means algorithm with application in medical image segmentation. Artif. Intell. Med. 32(1), 37–50 (2004)CrossRefGoogle Scholar
  36. 36.
    Du, W., Inoue, K., Urahama, K.: Robust kernel fuzzy clustering. In: Lecture Notes on Artificial Intelligence, vol. 3613, pp. 454–461. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  37. 37.
    Filippone, M., Camastra, F., Masulli, F., Rovetta, S.: A survey of kernel and spectral methods for clustering. Pattern Recogn. 41, 176–190 (2008)CrossRefGoogle Scholar
  38. 38.
    Havens, T.S., Bezdek, J.C., Palaniswami, M.: Incremental kernel fuzzy c-means. In: Madani, K., et al. (eds.) Computational Intelligence, SCI, vol. 399, pp. 3–18. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  39. 39.
    Krishnapuram, R., Keller, J.M.: A possibilistic approach to clustering. IEEE Trans. Fuzzy Syst. 1(2), 98–110 (1993)CrossRefGoogle Scholar
  40. 40.
    Krishnapuram, R., Keller, J.M.: The possibilistic c-means algorithm: insights and recommendations. IEEE Trans. Fuzzy Syst. 4(3), 385–393 (1996)CrossRefGoogle Scholar
  41. 41.
    Bodyanskiy, Y., Tyshchenko, O., Kopaliani, D.: An evolving neuro-fuzzy system for online fuzzy clustering. In: Proceedings of the International Conference on Computer Sciences and Information Technologies, CSIT 2015, pp. 158–161, Lviv (2015)Google Scholar
  42. 42.
    Hu, Z., Bodyanskiy, Y.V., Tyshchenko, O.K., Samitova, V.O.: Possibilistic fuzzy clustering for categorical data arrays based on frequency prototypes and dissimilarity measures. Int. J. Intell. Syst. Appl. (IJISA) 9(5), 55–61 (2017).  https://doi.org/10.5815/ijisa.2017.05.07CrossRefGoogle Scholar
  43. 43.
    Hu, Z., Bodyanskiy, Y.V., Tyshchenko, O.K.: A deep cascade neuro-fuzzy system for high-dimensional online fuzzy clustering. In: Proceedings of the 2016 IEEE 1st International Conference on Data Stream Mining and Processing, DSMP 2016, pp. 318–322, Lviv (2016)Google Scholar
  44. 44.
    Hu, Z., Bodyanskiy, Y.V., Tyshchenko, O.K.: A cascade deep neuro-fuzzy system for high-dimensional online possibilistic fuzzy clustering. In: Proceedings of the 11th International Scientific and Technical Conference “Computer Sciences and Information Technologies”, CSIT 2016, pp. 119–122, Lviv (2016)Google Scholar
  45. 45.
    Aggarwal, C.C.: Data Streams: Models and Algorithms. Springer Science & Business Media, Berlin (2007)CrossRefGoogle Scholar
  46. 46.
    Bifet, A.: Adaptive Stream Mining: Pattern Learning and Mining from Evolving Data Streams. IOS Press, Amsterdam (2010)MATHGoogle Scholar
  47. 47.
    Bodyanskiy, Y.V., Tyshchenko, O.K., Kopaliani, D.S.: An evolving connectionist system for data stream fuzzy clustering and its online learning. Neurocomputing 262, 41–56 (2017)CrossRefGoogle Scholar
  48. 48.
    Hu, Z., Bodyanskiy, Y.V., Tyshchenko, O.K., Boiko, O.O.: A neuro-fuzzy Kohonen network for data stream possibilistic clustering and its online self-learning procedure. Appl. Soft Comput. J. (2017, in Press).  https://doi.org/10.1016/j.asoc.2017.09.042
  49. 49.
    Zahirniak, D., Chapman, R., Rogers, S., Suter, B., Kabrisky, M., Piati, V.: Pattern recognition using radial basis function network. In: Proceedings of 6th Annual Aerospace Application of Artificial Intelligence Conference, pp. 249–260, Dayton, OH (1990)Google Scholar
  50. 50.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)CrossRefGoogle Scholar
  51. 51.
    Cortes, C., Vapnik, V.: Support vector networks. Mach. Learn. 20, 273–297 (1995)MATHGoogle Scholar
  52. 52.
    Specht, D.F.: A general regression neural network. IEEE Trans. Neural Netw. 2, 568–576 (1991)CrossRefGoogle Scholar
  53. 53.
    Rooki, R.: Application of general regression neural network (GRNN) for indirect measuring pressure loss of Herschel-Bulkley drilling fluids in oil drilling. Measurement 85, 184–191 (2016)CrossRefGoogle Scholar
  54. 54.
    Alilou, V.K., Yaghmaee, F.: Application of GRNN neural network in non-texture image inpainting and restoration. Pattern Recogn. Lett. 62, 24–31 (2015)CrossRefGoogle Scholar
  55. 55.
    Bodyanskiy, Y., Kolodyazhniy, V., Stephan, A.: Recursive fuzzy clustering algorithms. In: Proceedings of 10th East West Fuzzy Colloqium, pp. 276–283, Zittau/Görlitz (2002)Google Scholar
  56. 56.
    Bodyanskiy, Y.: Computational intelligence techniques for data analysis. In: Lecture Notes in Informatics, vol. P.72, pp. 15–36, Bonn, GI (2005)Google Scholar
  57. 57.
    Gorshkov, Y., Kolodyazhniy, V., Bodyanskiy, Y.: New recursive learning algorithm for fuzzy Kohonen clustering network. In: Proceedings of 17th International Workshop on Nonlinear Dynamics of Electronic Systems, pp. 58–61, Rapperswil, Switzerland (2009)Google Scholar
  58. 58.
    Park, D.C., Dagher, I.: Gradient based fuzzy c-means (GBFCM) algorithm. In: Proceedings of IEEE International Conference on Neural Networks, pp. 1626–1631 (1984)Google Scholar
  59. 59.
    Chung, F.-L., Lee, T.: Unsupervised fuzzy competitive learning with monotonically decreasing fuzziness. In: Proceedings of 1993 International Joint Conference on Neural Networks, pp. 2929–2932 (1993)Google Scholar
  60. 60.
    Klawonn, F., Höppner, F.: What is fuzzy about fuzzy clustering? Understanding and improving the concept of the fuzzifier. In: Lecture Notes in Computer Science, vol. 2811, pp. 254–264. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  61. 61.
    Kolchygin, B., Bodyanskiy, Y.: Adaptive fuzzy clustering with a variable fuzzifier. Cybern. Syst. Anal. 49(3), 176–181 (2013)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Central China Normal UniversityWuhanChina
  2. 2.Kharkiv National University of Radio ElectronicsKharkivUkraine
  3. 3.Institute for Research and Applications of Fuzzy Modeling, CE IT4InnovationsUniversity of OstravaOstravaCzech Republic

Personalised recommendations