Fuzzy Entropy Used for Predictive Analytics

  • Christer CarlssonEmail author
  • Markku Heikkilä
  • József Mezei
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 341)


Process interruptions in (very) large production systems are difficult to deal with. Modern processes are highly automated; data is collected with sensor technology that forms a big data context and offers challenges to identify coming failures from the very large sets of data. The sensors collect huge amounts of data but the failure events are few and infrequent and hard to find (and even harder to predict). In this article, our goal is to develop models for predictive maintenance in a big data environment. The purpose of feature selection in the context of predictive maintenance is to identify a small set of process diagnostics that are sufficient to predict future failures. We apply interval-valued fuzzy sets and various entropy measures defined on them to perform feature selection on process diagnostics. We show how these models can be utilized as the basis of decision support systems in process industries to aid predictive maintenance.


Predictive analytics Fuzzy entropy Feature selection Failure prediction 



This research has been funded through the TEKES strategic research project Data to Intelligence [D2I], project number: 340/12.


  1. 1.
    Beath, C., Becerra-Fernandez, I., Ross, J., Short, J.: Finding value in the information explosion. MIT Sloan Manage. Rev. 53(4), 18 (2012)Google Scholar
  2. 2.
    Bolón-Canedo, V., Sánchez-Maroño, N., Alonso-Betanzos, A.: Recent advances and emerging challenges of feature selection in the context of big data. Knowl. Based Syst. 86, 33–45 (2015)CrossRefGoogle Scholar
  3. 3.
    Burillo, P., Bustince, H.: Entropy on intuitionistic fuzzy sets and on interval-valued fuzzy sets. Fuzzy Sets Syst. 78(3), 305–316 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Carlsson, C., Fullér, R.: Fuzzy Reasoning in Decision Making and Optimization, vol. 82. Springer Science & Business Media (2002)Google Scholar
  5. 5.
    Carlsson, C., Heikkilä, M., Mezei, J.: Possibilistic Bayes modelling for predictive analytics. In: 2014 IEEE 15th International Symposium on Computational Intelligence and Informatics (CINTI), pp. 15–20. IEEE (2014)Google Scholar
  6. 6.
    Chen, H., Chiang, R.H., Storey, V.C.: Business intelligence and analytics: from big data to big impact. MIS Q. 36(4), 1165–1188 (2012)Google Scholar
  7. 7.
    Dash, M., Liu, H.: Feature selection for classification. Intel. Data Anal. 1(3), 131–156 (1997)CrossRefGoogle Scholar
  8. 8.
    Davenport, T.H., Harris, J.G.: Competing on Analytics: The New Science of Winning. Harvard Business Press (2007)Google Scholar
  9. 9.
    De Luca, A., Termini, S.: A definition of a nonprobabilistic entropy in the setting of fuzzy sets theory. Inf. Control 20(4), 301–312 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    De Mántaras, R.L.: A distance-based attribute selection measure for decision tree induction. Mach. Learn. 6(1), 81–92 (1991)CrossRefGoogle Scholar
  11. 11.
    Demirkan, H., Delen, D.: Leveraging the capabilities of service-oriented decision support systems: putting analytics and big data in cloud. Dec. Supp. Syst. 55(1), 412–421 (2013)CrossRefGoogle Scholar
  12. 12.
    Diamantoulakis, P.D., Kapinas, V.M., Karagiannidis, G.K.: Big data analytics for dynamic energy management in smart grids. Big Data Res. 2(3), 94–101 (2015)CrossRefGoogle Scholar
  13. 13.
    Gandomi, A., Haider, M.: Beyond the hype: big data concepts, methods, and analytics. Int. J. Inf. Manage. 35(2), 137–144 (2015)CrossRefGoogle Scholar
  14. 14.
    Gass, S.I., Harris, C.M. (eds.): Encyclopedia of Operations Research and Management Science. Kluwer Academic Publishers, Dordrecht (1996)Google Scholar
  15. 15.
    Gil-Aluja, J.: Fuzzy Sets in the Management of Uncertainty. Springer Science & Business Media (2004)Google Scholar
  16. 16.
    Gorzałczany, M.B.: A method of inference in approximate reasoning based on interval-valued fuzzy sets. Fuzzy Sets Syst 21(1), 1–17 (1987)Google Scholar
  17. 17.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)zbMATHGoogle Scholar
  18. 18.
    Hall, M.A.: Correlation-based feature selection for machine learning. Ph.D. thesis, The University of Waikato (1999)Google Scholar
  19. 19.
    Hitzler, P., Janowicz, K.: Linked data, big data, and the 4th paradigm. Seman. Web 4(3), 233–235 (2013)Google Scholar
  20. 20.
    Janke, A.T., Overbeek, D.L., Kocher, K.E., Levy, P.D.: Exploring the potential of predictive analytics and big data in emergency care. Ann. Emerg. Med. (2015)Google Scholar
  21. 21.
    Jiang, Y., Tang, Y., Liu, H., Chen, Z.: Entropy on intuitionistic fuzzy soft sets and on interval-valued fuzzy soft sets. Inf. Sci. 240, 95–114 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Knopfmacher, J.: On measures of fuzziness. J. Math. Anal. Appl. 49(3), 529–534 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Kosko, B.: Fuzzy entropy and conditioning. Inf. Sci. 40(2), 165–174 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Kwak, N., Choi, C.-H.: Input feature selection for classification problems. IEEE Trans. Neural Netw. 13(1), 143–159 (2002)CrossRefGoogle Scholar
  25. 25.
    Lee, H.-M., Chen, C.-M., Chen, J.-M., Jou, Y.-L.: An efficient fuzzy classifier with feature selection based on fuzzy entropy. IEEE Trans. Syst. Man Cybern. Part B Cybern. 31(3), 426–432 (2001)CrossRefGoogle Scholar
  26. 26.
    Lee, S.-H., Kim, S., Kim, J.-M., Choi, C., Kim, J., Lee, S., Oh, Y.: Extraction of induction motor fault characteristics in frequency domain and fuzzy entropy. In: 2005 IEEE International Conference on Electric Machines and Drives, pp. 35–40. IEEE (2005)Google Scholar
  27. 27.
    Li, P., Liu, B.: Entropy of credibility distributions for fuzzy variables. IEEE Trans. Fuzzy Syst. 1(16), 123–129 (2008)Google Scholar
  28. 28.
    Li, J., Deng, G., Li, H., Zeng, W.: The relationship between similarity measure and entropy of intuitionistic fuzzy sets. Inf. Sci. 188, 314–321 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Liu, X.: Entropy, distance measure and similarity measure of fuzzy sets and their relations. Fuzzy Sets Syst. 52(3), 305–318 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Liu, B.: Uncertainty Theory. Springer, Berlin (2007)CrossRefzbMATHGoogle Scholar
  31. 31.
    Liu, H., Yu, L.: Toward integrating feature selection algorithms for classification and clustering. IEEE Knowl. Data Eng. 17(4), 491–502 (2005)CrossRefGoogle Scholar
  32. 32.
    McAfee, A., Brynjolfsson, E.: Big data: the management revolution. Harvard Bus. Rev. 90(10), 60–68 (2012)Google Scholar
  33. 33.
    Mezei, J., Morente-Molinera, J.A., Carlsson, C.: Feature selection with fuzzy entropy to find similar cases. In: Advance Trends in Soft Computing, pp. 383–390. Springer, Berlin (2014)Google Scholar
  34. 34.
    Morabito, V.: Big data driven business models. In: Big Data and Analytics, pp. 65–80. Springer, Berlin (2015)Google Scholar
  35. 35.
    Muir, B.M., Moray, N.: Trust in automation. Part ii. Experimental studies of trust and human intervention in a process control simulation. Ergonomics 39(3), 429–460 (1996)CrossRefGoogle Scholar
  36. 36.
    Nieto-Santisteban, M.A., Szalay, A.S., Thakar, A.R., O’Mullane, W.J., Gray, J., Annis, J.: When database systems meet the grid. arXiv preprint cs/0502018 (2005)Google Scholar
  37. 37.
    Papermaking—Parts 1–3. Paperi ja Puu Oy, Jyväskylä (2007)Google Scholar
  38. 38.
    Parthalain, N., Jensen, R., Shen, Q.: Fuzzy entropy-assisted fuzzy-rough feature selection. In: 2006 IEEE International Conference on Fuzzy Systems, pp. 423–430. IEEE (2006)Google Scholar
  39. 39.
    Qin, S.J.: Process data analytics in the era of big data. AIChE J. 60(9), 3092–3100 (2014)CrossRefGoogle Scholar
  40. 40.
    R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2014)Google Scholar
  41. 41.
    Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27(3), 379–423 (1948)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Shie, J.-D., Chen, S.-M.: Feature subset selection based on fuzzy entropy measures for handling classification problems. Appl. Intell. 28(1), 69–82 (2008)CrossRefGoogle Scholar
  43. 43.
    Szmidt, E., Kacprzyk, J.: Entropy for intuitionistic fuzzy sets. Fuzzy Set. Syst. 118(3), 467–477 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Tao, W.-B., Tian, J.-W., Liu, J.: Image segmentation by three-level thresholding based on maximum fuzzy entropy and genetic algorithm. Patt. Rec. Lett. 24(16), 3069–3078 (2003)CrossRefGoogle Scholar
  45. 45.
    Wu, D., Mendel, J.M.: Uncertainty measures for interval type-2 fuzzy sets. Inf. Sci. 177(23), 5378–5393 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Wu, X., Gu, H., Hu, X., Dong, Y.: Application of the fuzzy entropy weight in risk assessment of the engineering project. In: Fifth International Conference on Information Assurance and Security, vol. 1, pp. 145–148. IEEE (2009)Google Scholar
  47. 47.
    Xu, Z., Xia, M.: Hesitant fuzzy entropy and cross-entropy and their use in multiattribute decision-making. Int. J. Intell. Syst. 27(9), 799–822 (2012)CrossRefGoogle Scholar
  48. 48.
    Yager, R.R.: A procedure for ordering fuzzy subsets of the unit interval. Inf. Sci. 24(2), 143–161 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  49. 49.
    Ye, J.: Fault diagnosis of turbine based on fuzzy cross entropy of vague sets. Expert Syst. Appl. 36(4), 8103–8106 (2009)CrossRefGoogle Scholar
  50. 50.
    Zadeh, L.A.: Fuzzy sets. Inform. Cont. 8(3), 338–353 (1965)MathSciNetCrossRefzbMATHGoogle Scholar
  51. 51.
    Zeng, W., Li, H.: Relationship between similarity measure and entropy of interval valued fuzzy sets. Fuzzy Sets Syst. 157(11), 1477–1484 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  52. 52.
    Zhai, Y., Ong, Y.-S., Tsang, I.W.: The emerging “big dimensionality”. IEEE Comput. Intell. Mag. 9(3), 14–26 (2014)CrossRefGoogle Scholar
  53. 53.
    Zhang, Q.-S., Jiang, S.-Y.: A note on information entropy measures for vague sets and its applications. Inf. Sci. 178(21), 4184–4191 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  54. 54.
    Zikopoulos, P., Eaton, C.: Understanding big data: analytics for enterprise class hadoop and streaming data. McGraw-Hill Osborne Media, New York (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Christer Carlsson
    • 1
    Email author
  • Markku Heikkilä
    • 1
  • József Mezei
    • 1
  1. 1.IAMSRÅbo Akademi UniversityTurkuFinland

Personalised recommendations