Impact of Fuzziness Measures on the Performance of Semi-supervised Learning

  • Muhammed J. A. PatwaryEmail author
  • Xi-Zhao Wang
  • Dasen Yan


Usage of fuzziness in the study of semi-supervised learning is relatively new. In this study, the divide-and-conquer strategy is used to investigate the performance of semi-supervised learning. To this end, testing dataset is divided into three categories, namely low, medium and high-fuzzy samples based on the magnitude of fuzziness of each sample. It is experimentally confirmed that if the low-fuzzy samples are added from the testing dataset to the original training dataset and the model is retrained, then the accuracy can be improved. To measure the amount of fuzziness of each sample, four different fuzziness measuring models are used in this study. Experimental results support that improvement of accuracy is dependent on which fuzziness measuring model is used to measure the fuzziness of each sample. Wilcoxon signed-rank test shows that choosing a specific fuzziness measuring model is significant or not. Finally, from the Wilcoxon signed-rank test, the best model is chosen, which can be used along with semi-supervised learning to improve its performance.


Fuzziness Semi-supervised learning Divide-and-conquer strategy Measures of fuzziness Fuzzy classifier Wilcoxon signed-rank test 



This work was supported in part by the National Natural Science Foundation of China (Grants 61772344, 61732011, 71371063 and 61811530324) and in part by Basic Research Project of Knowledge Innovation Program in ShenZhen (JCYJ20180305125850156).


  1. 1.
    Zhu, X., Goldberg, A.: Introduction to semi-supervised learning. Synth. Lect. Artif. Intell. Mach. Learn. 3(1), 1–130 (2009)CrossRefzbMATHGoogle Scholar
  2. 2.
    Seeger, M.: Learning with Labeled and Unlabeled Data (Tech. Rep.). Edinburgh, UK: Institute for Adaptive and Neural Computation, University of Edinburgh (2000)Google Scholar
  3. 3.
    Chawla, N.V., Karakoulas, G.: Learing from labeled and unlabeled data: an empirical study across techniques and domain. J. Artif. Intell. Res. 23, 331–366 (2005)CrossRefzbMATHGoogle Scholar
  4. 4.
    Zhou, Z.-H., Zhan, D.-C., Yang, Q.: Semi-supervised learning with very few labeled training examples. In: AAAI, pp. 675–680 (2007)Google Scholar
  5. 5.
    Zadeh, L.A.: Probability measures of fuzzy events. J. Math. Anal. Appl. 23(2), 421–427 (1968)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Miyato, T., Maeda, S., Ishii, S., Koyama, M.: Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. (2018). Google Scholar
  7. 7.
    Li, C., Zhu, J., Zhang, B.: Max-margin deep generative models for (semi-) supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 40(11), 2762–2775 (2018)CrossRefGoogle Scholar
  8. 8.
    Pratama, M., Lughofer, E., Lim, C.P., Rahayu, W., Dillon, T., Budiyono, A.: pClass+: a novel evolving semi-supervised classifier. Int. J. Fuzzy Syst. 19(3), 863–880 (2017)CrossRefGoogle Scholar
  9. 9.
    Zhou, W., Qiao, S., Yi, Y., Han, N., Chen, Y., Lei, G.: Automatic optic disc detection using low-rank representation based semi-supervised extreme learning machine. Int. J. Mach. Learn. Cybern. (2019). Google Scholar
  10. 10.
    Sang, N., Gan, H., Fan, Y., Wu, W., Yang, Z.: Adaptive safety degree-based safe semi-supervised learning. Int. J. Mach. Learn. Cybern. 10(5), 1101–1108 (2019)CrossRefGoogle Scholar
  11. 11.
    Ashfaq, R.A.R., Wang, X.-Z., Huang, J.Z., Abbas, H., He, Y.-L.: Fuzziness based semi-supervised learning approach for intrusion detection system. Inf. Sci. 378, 484–497 (2017)CrossRefGoogle Scholar
  12. 12.
    Luca, A., Termini, S.: A definition of a nonprobabilistic entropy in the setting of fuzzy sets theory. Inf. Control 20(4), 301–312 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Patwary, M.J.A., Akter, S., Mahmud, T.: An expert system to detect uterine cancer under uncertainty, IOSR J. Comput. Eng. (IOSR-JCE) 16(5), 36–47 (2014)CrossRefGoogle Scholar
  14. 14.
    Patwary, M.J.A., Rahman, M.O., Hossain, M.S.: Uncertainty handling in ship assessment: a case study of Bangladesh. J. Invest. Manag. 4(5), 152–161 (2015)CrossRefGoogle Scholar
  15. 15.
    Sanchez, D., Trillas, E.: Measures of fuzziness under different uses of fuzzy sets. In: International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, pp. 25–34. Springer (2012)Google Scholar
  16. 16.
    Ralescu, D., Adams, G.: The fuzzy integral. J. Math. Anal. Appl. 75(2), 562–570 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Wang, Z.-X.: Fuzzy measures and measures of fuzziness. J. Math. Anal. Appl. 104(2), 589–601 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Farhadinia, B., Xu, Z.: Entropy measures for hesitant fuzzy sets and their extensions. In: Information Measures for Hesitant Fuzzy Sets and Their Extensions. Uncertainty and Operations Research, pp. 69-102. Springer, Singapore (2019)CrossRefGoogle Scholar
  19. 19.
    Jozsef, D., Lorant, P.: Measures of fuzziness. Ann. Universitasis Scientiarium Budapestinensis Sect. Computatorica 12, 69–78 (1991)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Li, W., Pedrycz, W., Xue, X., Xu, W., Fan, B.: Fuzziness and incremental information of disjoint regions in double-quantitative decision-theoretic rough set model. Int. J. Mach. Learn. Cybern. (2018). Google Scholar
  21. 21.
    Kaufmann, A.: Introduction to the Theory of Fuzzy Subsets, vol. 1. Academic Press, Cambridge (1975)zbMATHGoogle Scholar
  22. 22.
    Ebanks, B.R.: On measures of fuzziness and their representations. J. Math. Anal. Appl. 94(1), 24–37 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Scudder, H.: Probability of error of some adaptive pattern-recognition machines. IEEE Trans. Inf. Theory 11(3), 363–371 (1965)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Ma, M., Deng, T., Wang, N., Chen, Y.: Semi-supervised rough fuzzy Laplacian Eigenmaps for dimensionality reduction. Int. J. Mach. Learn. Cybernet. 10(2), 397–411 (2019)CrossRefGoogle Scholar
  25. 25.
    Wei, S., Li, Z., Zhang, C.: Combined constraint-based with metric-based in semi-supervised clustering ensemble. Int. J. Mach. Learn. Cybernet. 9(7), 1085–1100 (2018)CrossRefGoogle Scholar
  26. 26.
    Huang, R., Zhang, G., Chen, J.: Semi-supervised discriminant Isomap with application to visualization, image retrieval and classification. Int. J. Mach. Learn. Cybern. 10(6), 1269–1278 (2019)CrossRefGoogle Scholar
  27. 27.
    Uddin, M.T., Patwary, M.J.A., Ahsan, T., Alam, M.S.: Predicting the popularity of online news from content metadata. In: International Conference on Innovations in Science, Engineering and Technology (ICISET), pp. 1–5 (2016)Google Scholar
  28. 28.
    Patwary, M.J.A., Hossain, S.: Risk analysis of buildings using an expert system: a case study in Bangladesh. In: International Conference on Advances in Electrical Engineering (ICAEE), pp. 141–144 (2015)Google Scholar
  29. 29.
    Yu, Y., Ji, Z., Li, X., Guo, J., Zhang, Z., Ling, H., Wu, F.: Transductive zero-shot learning with a self-training dictionary approach. IEEE Trans. Cybernet. 99, 1–12 (2018)Google Scholar
  30. 30.
    Wu, D., Shang, M., Luo, X., Xu, J., Yan, H., Deng, W., Wang, G.: Self-training semi-supervised classification based on density peaks of data. Neurocomputing 275, 180–191 (2018)CrossRefGoogle Scholar
  31. 31.
    Yarowsky, D.: Unsupervised word sense disambiguation rivaling supervised methods. In: Proceedings of the 33rd annual meeting on Association for Computational Linguistics, Association for Computational Linguistics, pp. 189–196 (1995)Google Scholar
  32. 32.
    Riloff, E., Wiebe, J., Wilson, T.: Learning subjective nouns using extraction pattern bootstrapping. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003-Volume 4, Association for Computational Linguistics, pp. 25–32 (2003)Google Scholar
  33. 33.
    Abdelgayed, T.S., Morsi, W.G., Sidhu, T.S.: Fault detection and classification based on co-training of semisupervised machine learning. IEEE Trans. Ind. Electron. 65(2), 1595–1605 (2018)CrossRefGoogle Scholar
  34. 34.
    Hu, T., Huang, X., Li, J., Zhang, L.: A novel co-training approach for urban land cover mapping with unclear Landsat time series imagery. Remote Sens. Environ. 217, 144–157 (2018)CrossRefGoogle Scholar
  35. 35.
    Maeireizo, B., Litman, D., Hwa, R.: Co-training for predicting emotions with spoken dialogue data. In: Proceedings of the ACL 2004 on Interactive Poster and Demonstration Sessions, Association for Computational Linguistics, p. 28 (2004)Google Scholar
  36. 36.
    Nie, F., Cai, G., Li, J., Li, X.: Auto-weighted multi-view learning for image clustering and semi-supervised classification. IEEE Trans. Image Process. 27(3), 1501–1511 (2018)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Jing, P., Su, Y., Nie, L., Bai, X., Liu, J., Wang, M.: Low-rank multi-view embedding learning for micro-video popularity prediction. IEEE Trans. Knowl. Data Eng. 30(8), 1519–1532 (2018)CrossRefGoogle Scholar
  38. 38.
    Zhou, Z.-H., Li, M.: Tri-training: exploiting unlabeled data using three classifiers. IEEE Trans. Knowl. Data Eng. 17(11), 1529–1541 (2005)CrossRefGoogle Scholar
  39. 39.
    Kim, D., Seo, D., Cho, S., Kang, P.: Multi-co-training for document classification using various document representations: TF-IDF, LDA, and Doc2Vec. Inf. Sci. 477, 15–29 (2019)CrossRefGoogle Scholar
  40. 40.
    Li, D., Dick, S.: Residential household non-intrusive load monitoring via graph-based multi-label semi-supervised learning. IEEE Trans. Smart Grid (2018). Google Scholar
  41. 41.
    Gan, H., Li, Z., Wu, W., Luo, Z., Huang, R.: Safety-aware graph-based semi-supervised learning. Expert Syst. Appl. 107, 243–254 (2018)CrossRefGoogle Scholar
  42. 42.
    Zhou, D., Huang, J., Scholkopf, B.: Learning from labeled and unlabeled data on a directed graph. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 1036–1043. ACM (2005)Google Scholar
  43. 43.
    Blum, A., Chawla, S.: Learning from labeled and unlabeled data using graph mincuts [C]. In: Proceedings of the Eighteenth International Conference on Machine Learning (ICML 2001), Morgan Kaufmann Publishers Inc., Williams College, Williamstown, MA, USA, June 28–July 1 (2001)Google Scholar
  44. 44.
    Wang, X., He, Y.: Learning from uncertainty for big data: future analytical challenges and strategies. IEEE Syst. Man Cybernet. Mag. 2(2), 26–31 (2016)CrossRefGoogle Scholar
  45. 45.
    Patwary, M.J.A., Wang, X.-Z.: Sensitivity analysis on initial classifier accuracy in fuzziness based semi-supervised learning. Inf. Sci. 490, 93–112 (2019)CrossRefGoogle Scholar
  46. 46.
    Wang, X.-Z., Ashfaq, R.A.R., Fu, A.-M.: Fuzziness based sample categorization for classifier performance improvement. J. Intell. Fuzzy Syst. 29(3), 1185–1196 (2015)MathSciNetCrossRefGoogle Scholar
  47. 47.
    Keller, J.M., Gray, M.R., Givens, J.A.: A fuzzy k-nearest neighbor algorithm. IEEE Trans. Syst. Man Cybernet. 4, 580–585 (1985)CrossRefGoogle Scholar
  48. 48.
    Huang, G.-B., Chen, L., Siew, C.K., et al.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17(4), 879–892 (2006)CrossRefGoogle Scholar
  49. 49.
    Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: theory and applications. Neurocomputing 70(1–3), 489–501 (2006)CrossRefGoogle Scholar
  50. 50.
    Huang, G.-B., Wang, D.H., Lan, Y.: Extreme learning machines: a survey. Int. J. Mach. Learn. Cybernet. 2(2), 107–122 (2011)CrossRefGoogle Scholar
  51. 51.
    Liu, J., Patwary, M.J., Sun, X., Tao, K.: An experimental study on symbolic extreme learning machine. Int. J. Mach. Learn. Cybernet. 10(4), 787–797 (2019)CrossRefGoogle Scholar
  52. 52.
    Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1(6), 80–83 (1945)CrossRefGoogle Scholar
  53. 53.
    Demsar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  54. 54.
    Frank, A.: Uci machine learning repository. University of California, School of Information and Computer Science (2010). Accessed 10 Dec 2018

Copyright information

© Taiwan Fuzzy Systems Association 2019

Authors and Affiliations

  • Muhammed J. A. Patwary
    • 1
    Email author
  • Xi-Zhao Wang
    • 1
  • Dasen Yan
    • 1
  1. 1.Big Data Institute, College of Computer Science and Software EngineeringKey Guangdong of Intelligent Laboratory Information Processing Shenzhen UniversityShenzhenChina

Personalised recommendations