Advertisement

Neurofuzzy Systems

  • Ke-Lin DuEmail author
  • M. N. S. Swamy
Chapter

Abstract

Hybridization of fuzzy logic and neural networks yields neurofuzzy systems, which capture the merits of both paradigms. This chapter first describes how to extract rules from neural networks and data, and then introduces how the synergy of fuzzy logic and neural network paradigms is implemented.

References

  1. 1.
    Abe, S., & Lan, M. S. (1995). Fuzzy rules extraction directly from numerical data for function approximation. IEEE Transactions on Systems, Man, and Cybernetics, 25(1), 119–129.Google Scholar
  2. 2.
    Angulo, C., Anguita, D., Gonzalez-Abril, L., & Ortega, J. A. (2008). Support vector machines for interval discriminant analysis. Neurocomputing, 71, 1220–1229.CrossRefGoogle Scholar
  3. 3.
    Azeem, M. F., Hanmandlu, M., & Ahmad, N. (2000). Generalization of adaptive neuro-fuzzy inference systems. IEEE Transactions on Neural Networks, 11(6), 1332–1346.CrossRefGoogle Scholar
  4. 4.
    Baker, M. R., & Patil, R. B. (1998). Universal approximation theorem for interval neural networks. Reliable Computing, 4, 235–239.MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Batuwita, R., & Palade, V. (2010). FSVM-CIL: Fuzzy support vector machines for class imbalance learning. IEEE Transactions on Fuzzy Systems, 18(3), 558–571.CrossRefGoogle Scholar
  6. 6.
    Benitez, J. M., Castro, J. L., & Requena, I. (1997). Are artificial neural networks black boxes? IEEE Transactions on Neural Networks, 8(5), 1156–1164.CrossRefGoogle Scholar
  7. 7.
    Berenji, H. R., & Vengerov, D. (2003). A convergent actor–critic-based FRL algorithm with application to power management of wireless transmitters. IEEE Transactions on Fuzzy Systems, 11(4), 478–485.CrossRefGoogle Scholar
  8. 8.
    Castro, J. L., Mantas, C. J., & Benitez, J. M. (2002). Interpretation of artificial neural networks by means of fuzzy rules. IEEE Transactions on Neural Networks, 13(1), 101–116.CrossRefGoogle Scholar
  9. 9.
    Castro, J. L., Flores-Hidalgo, L. D., Mantas, C. J., & Puche, J. M. (2007). Extraction of fuzzy rules from support vector machines. Fuzzy Sets and Systems, 158, 2057–2077.MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Cechin, A., Epperlein, U., Koppenhoefer, B., & Rosenstiel, W. (1996). The extraction of Sugeno fuzzy rules from neural networks. In M. Verleysen (Ed.), Proceedings of the European Symposium on Artificial Neural Networks (pp. 49–54). Bruges, Belgium.Google Scholar
  11. 11.
    Chen, J. L., & Chang, J. Y. (2000). Fuzzy perceptron neural networks for classifiers with numerical data and linguistic rules as inputs. IEEE Transactions on Fuzzy Systems, 8(6), 730–745.CrossRefGoogle Scholar
  12. 12.
    Chen, M. S., & Liou, R. J. (1999). An efficient learning method of fuzzy inference system. In Proceedings of IEEE International Fuzzy Systems (pp. 634–638). Seoul, Korea.Google Scholar
  13. 13.
    Chen, Z.-P., Jiang, J.-H., Li, Y., Liang, Y.-Z., & Yu, R.-Q. (1999). Fuzzy linear discriminant analysis for chemical datasets. Chemometrics and Intelligent Laboratory Systems, 45, 295–302.CrossRefGoogle Scholar
  14. 14.
    Chen, C. L. P., Zhang, C.-Y., Chen, L., & Gan, M. (2015). Fuzzy restricted Boltzmann machine for the enhancement of deep learning. IEEE Transactions on Fuzzy Systems, 23(6), 2163–2173.CrossRefGoogle Scholar
  15. 15.
    Chiang, J. H., & Hao, P. Y. (2004). Support vector learning mechanism for fuzzy rule-based modeling: A new approach. IEEE Transactions on Fuzzy Systems, 12(1), 1–12.CrossRefGoogle Scholar
  16. 16.
    Chuang, C.-C. (2007). Fuzzy weighted support vector regression with a fuzzy partition. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 37(3), 630–640.CrossRefGoogle Scholar
  17. 17.
    Denoeux, T., & Masson, M. H. (2004). Principal component analysis of fuzzy data using autoassociative neural networks. IEEE Transactions on Fuzzy Systems, 12(3), 336–349.CrossRefGoogle Scholar
  18. 18.
    Derhami, V., Majd, V. J., & Ahmadabadi, M. N. (2008). Fuzzy Sarsa learning and the proof of existence of its stationary points. Asian Journal of Control, 10(5), 535–549.MathSciNetCrossRefGoogle Scholar
  19. 19.
    Derhami, V., Majd, V. J., & Ahmadabadi, M. N. (2010). Exploration and exploitation balance management in fuzzy reinforcement learning. Fuzzy Sets and Systems, 161, 578–595.MathSciNetCrossRefGoogle Scholar
  20. 20.
    Duch, W. (2005). Uncertainty of data, fuzzy membership functions, and multilayer perceptrons. IEEE Transactions on Neural Networks, 16(1), 10–23.CrossRefGoogle Scholar
  21. 21.
    Feng, S., Chen, C. L. P., & Zhang, C.-Y. (2019). A fuzzy deep model based on fuzzy restricted boltzmann machines for high-dimensional data classification. IEEE Transactions on Fuzzy Systems.  https://doi.org/10.1109/TFUZZ.2019.2902111 (in press).
  22. 22.
    Fung, G., Sandilya, S., & Rao, R. (2005). Rule extraction from linear support vector machines. In Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining (KDD) (pp. 32–40).Google Scholar
  23. 23.
    Gabrays, B., & Bargiela, A. (2000). General fuzzy min–max neural networks for clustering and classification. IEEE Transactions on Neural Networks, 11(3), 769–783.CrossRefGoogle Scholar
  24. 24.
    Gallant, S. I. (1988). Connectionist expert systems. Communications of the ACM, 31(2), 152–169.CrossRefGoogle Scholar
  25. 25.
    Guillaume, S. (2001). Designing fuzzy inference systems from data: An interpretability-oriented review. IEEE Transactions on Fuzzy Systems, 9(3), 426–443.CrossRefGoogle Scholar
  26. 26.
    Hao, P.-Y. (2008). Fuzzy one-class support vector machines. Fuzzy Sets and Systems, 159, 2317–2336.MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Hayashi, Y., Buckley, J. J., & Czogala, E. (1993). Fuzzy neural network with fuzzy signals and weights. International Journal of Intelligent Systems, 8(4), 527–537.zbMATHCrossRefGoogle Scholar
  28. 28.
    Ho, D. W. C., Zhang, P. A., & Xu, J. (2001). Fuzzy wavelet networks for function learning. IEEE Transactions on Fuzzy Systems, 9(1), 200–211.CrossRefGoogle Scholar
  29. 29.
    Honda, K., Ichihashi, H., Ohue, M., & Kitaguchi, K. (2000). Extraction of local independent components using fuzzy clustering. In Proceedings of the 6th International Conference on Soft Computing (IIZUKA2000) (pp. 837–842).Google Scholar
  30. 30.
    Hwang, C., Hong, D. H., & Seok, K. H. (2006). Support vector interval regression machine for crisp input and output data. Fuzzy Sets and Systems, 157, 1114–1125.MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Ishibuchi, H., Tanaka, H., & Okada, H. (1993). An architecture of neural networks with interval weights and its application to fuzzy regression analysis. Fuzzy Sets and Systems, 57, 27–39.MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Ishikawa, M. (2000). Rule extraction by successive regularization. Neural Networks, 13(10), 1171–1183.CrossRefGoogle Scholar
  33. 33.
    Jacobsson, H. (2005). Rule extraction from recurrent neural networks: A taxonomy and review. Neural Computation, 17(6), 1223–1263.MathSciNetzbMATHCrossRefGoogle Scholar
  34. 34.
    Jang, J. S. R. (1993). ANFIS: Adaptive-network-based fuzzy inference systems. IEEE Transactions on Systems, Man, and Cybernetics, 23(3), 665–685.CrossRefGoogle Scholar
  35. 35.
    Jang, J. S. R., & Mizutani, E. (1996). Levenberg–Marquardt method for ANFIS learning. In Proceedings of Biennial International North American Fuzzy Information Processing (NAFIPS) (pp. 87–91). Berkeley, CA.Google Scholar
  36. 36.
    Jang, J. S. R., & Sun, C. I. (1993). Functional equivalence between radial basis function Networks and fuzzy inference systems. IEEE Transactions on Neural Networks, 4(1), 156–159.CrossRefGoogle Scholar
  37. 37.
    Jang, J. S. R., & Sun, C. I. (1995). Neuro-fuzzy modeling and control. Proceedings of the IEEE, 83(3), 378–406.CrossRefGoogle Scholar
  38. 38.
    Jeng, J.-T., Chuang, C.-C., & Su, S.-F. (2003). Support vector interval regression networks for interval regression analysis. Fuzzy Sets and Systems, 138, 283–300.MathSciNetzbMATHCrossRefGoogle Scholar
  39. 39.
    Jin, Y. (2003). Advanced fuzzy systems design and applications. Heidelberg, Germany: Physica-Verlag.zbMATHCrossRefGoogle Scholar
  40. 40.
    Jouffe, L. (1998). Fuzzy inference system learning by reinforcement methods. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 28(3), 338–355.CrossRefGoogle Scholar
  41. 41.
    Juang, C.-F., & Hsieh, C.-D. (2009). TS-fuzzy system-based support vector regression. Fuzzy Sets and Systems, 160, 2486–2504.MathSciNetzbMATHCrossRefGoogle Scholar
  42. 42.
    Juang, C.-F., Chiu, S.-H., & Chang, S.-W. (2007). A self-organizing TS-type fuzzy network with support vector learning and its application to classification problems. IEEE Transactions on Fuzzy Systems, 15(5), 998–1008.CrossRefGoogle Scholar
  43. 43.
    Keller, J. M., Gray, M. R., & Givens, J. A, Jr. (1985). A fuzzy \(K\)-nearest neighbor algorithm. IEEE Transactions on Systems, Man, and Cybernetics, 15(4), 580–585.CrossRefGoogle Scholar
  44. 44.
    Kim, J., & Kasabov, N. (1999). HyFIS: Adaptive neuro-fuzzy inference systems and their application to nonlinear dynamical systems. Neural Networks, 12, 1301–1319.CrossRefGoogle Scholar
  45. 45.
    Kolman, E., & Margaliot, M. (2005). Are artificial neural networks white boxes? IEEE Transactions on Neural Networks, 16(4), 844–852.CrossRefGoogle Scholar
  46. 46.
    Kolman, E., & Margaliot, M. (2009). Extracting symbolic knowledge from recurrent neural networks—A fuzzy logic approach. Fuzzy Sets and Systems, 160, 145–161.MathSciNetzbMATHCrossRefGoogle Scholar
  47. 47.
    Lin, C.-F., & Wang, S.-D. (2002). Fuzzy support vector machines. IEEE Transactions on Neural Networks, 13(2), 464–471.CrossRefGoogle Scholar
  48. 48.
    Lin, C.-T., Yeh, C.-M., Liang, S.-F., Chung, J.-F., & Kumar, N. (2006). Support-vector-based fuzzy neural network for pattern classification. IEEE Transactions on Fuzzy Systems, 14(1), 31–41.CrossRefGoogle Scholar
  49. 49.
    Liu, P. (2000). Max–min fuzzy Hopfield neural networks and an efficient learning algorithm. Fuzzy Sets and Systems, 112, 41–49.MathSciNetzbMATHCrossRefGoogle Scholar
  50. 50.
    Liu, P., & Li, H. (2004). Efficient learning algorithms for three-layer regular feedforward fuzzy neural networks. IEEE Transactions on Neural Networks, 15(3), 545–558.CrossRefGoogle Scholar
  51. 51.
    Liu, P., & Li, H. (2005). Hierarchical TS fuzzy system and its universal approximation. Information Sciences, 169, 279–303.MathSciNetzbMATHCrossRefGoogle Scholar
  52. 52.
    Liu, Y.-H., & Chen, Y.-T. (2007). Face recognition using total margin-based adaptive fuzzy support vector machines. IEEE Transactions on Neural Networks, 18(1), 178–192.CrossRefGoogle Scholar
  53. 53.
    Lou, S. T., & Zhang, X. D. (2003). Fuzzy-based learning rate determination for blind source separation. IEEE Transactions on Fuzzy Systems, 11(3), 375–383.CrossRefGoogle Scholar
  54. 54.
    Mitra, S., & Hayashi, Y. (2000). Neuro-fuzzy rule generation: Survey in soft computing framework. IEEE Transactions on Neural Networks, 11(3), 748–768.CrossRefGoogle Scholar
  55. 55.
    Mizutani, E., & Jang, J. S. (1995). Coactive neural fuzzy modeling. In Proceedings of IEEE International Conference on Neural Networks (Vol. 2, pp. 760–765). Perth, Australia.Google Scholar
  56. 56.
    Nauck, D., Klawonn, F., & Kruse, R. (1997). Foundations of neuro-fuzzy systems. New York: Wiley.zbMATHGoogle Scholar
  57. 57.
    Nicholls, J. G., Martin, A. R., & Wallace, B. G. (1992). From neuron to brain: A cellular and molecular approach to the function of the nervous system (3rd ed.). Sunderland, MA: Sinauer Associates.Google Scholar
  58. 58.
    Nikov, A., & Stoeva, S. (2001). Quick fuzzy backpropagation algorithm. Neural Networks, 14, 231–244.zbMATHCrossRefGoogle Scholar
  59. 59.
    Nunez, H., Angulo, C., & Catala, A. (2002). Rule extraction from support vector machines. In Proceedings of European Symposium on Artificial Neural Networks (pp. 107–112).Google Scholar
  60. 60.
    Nunez, H., Angulo, C., & Catala, A. (2006). Rule-based learning systems for support vector machines. Neural Processing Letters, 24, 1–18.CrossRefGoogle Scholar
  61. 61.
    Omlin, C. W., & Giles, C. L. (1996). Extraction of rules from discrete-time recurrent neural networks. Neural Networks, 9, 41–52.CrossRefGoogle Scholar
  62. 62.
    Omlin, C. W., Thornber, K. K., & Giles, C. L. (1998). Fuzzy finite-state automata can be deterministically encoded into recurrent neural networks. IEEE Transactions on Fuzzy Systems, 6, 76–89.CrossRefGoogle Scholar
  63. 63.
    Pedrycz, W., & Rocha, A. F. (1993). Fuzzy-set based models of neurons and knowledge-based networks. IEEE Transactions on Fuzzy Systems, 1(4), 254–266.CrossRefGoogle Scholar
  64. 64.
    Pedrycz, W., Reformat, M., & Li, K. (2006). OR/AND neurons and the development of interpretable logic models. IEEE Transactions on Neural Networks, 17(3), 636–658.CrossRefGoogle Scholar
  65. 65.
    Roque, A. M. S., Mate, C., Arroyo, J., & Sarabia, A. (2007). iMLP: Applying multi-layer perceptrons to interval-valued data. Neural Processing Letters, 25, 157–169.CrossRefGoogle Scholar
  66. 66.
    Simpson, P. K. (1992). Fuzzy min–max neural networks—Part I: Classification. IEEE Transactions on Neural Networks, 3, 776–786.CrossRefGoogle Scholar
  67. 67.
    Simpson, P. K. (1993). Fuzzy min–max neural networks—Part II: Clustering. IEEE Transactions on Fuzzy Systems, 1(1), 32–45.CrossRefGoogle Scholar
  68. 68.
    Sisman-Yilmaz, N. A., Alpaslan, F. N., & Jain, L. (2004). ANFIS-unfolded-in-time for multivariate time series forecasting. Neurocomputing, 61, 139–168.CrossRefGoogle Scholar
  69. 69.
    Soria-Olivas, E., Martin-Guerrero, J. D., Camps-Valls, G., Serrano-Lopez, A. J., Calpe-Maravilla, J., & Gomez-Chova, L. (2003). A low-complexity fuzzy activation function for artificial neural networks. IEEE Transactions on Neural Networks, 14(6), 1576–1579.CrossRefGoogle Scholar
  70. 70.
    Stoeva, S., & Nikov, A. (2000). A fuzzy backpropagation algorithm. Fuzzy Sets and Systems, 112, 27–39.MathSciNetzbMATHCrossRefGoogle Scholar
  71. 71.
    Sun, C. T. (1994). Rule-base structure identification in an adaptive-network-based inference system. IEEE Transactions on Fuzzy Systems, 2(1), 64–79.CrossRefGoogle Scholar
  72. 72.
    Sussner, P., & Valle, M. E. (2006). Implicative fuzzy associative memories. IEEE Transactions on Fuzzy Systems, 14(6), 793–807.CrossRefGoogle Scholar
  73. 73.
    Thawonmas, R., & Abe, S. (1999). Function approximation based on fuzzy rules extracted from partitioned numerical data. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 29(4), 525–534.CrossRefGoogle Scholar
  74. 74.
    Tsujinishi, D., & Abe, S. (2003). Fuzzy least squares support vector machines for multiclass problems. Neural Networks, 16, 785–792.CrossRefGoogle Scholar
  75. 75.
    Ultsch, A., Mantyk, R., & Halmans, G. (1993). Connectionist knowledge acquisition tool: CONKAT. In D. J. Hand (Ed.), Artificial intelligence frontiers in statistics: AI and statistics (Vol. 3, pp. 256–263). London: Chapman & Hall.CrossRefGoogle Scholar
  76. 76.
    Vuorimaa, P. (1994). Fuzzy self-organizing map. Fuzzy Sets and Systems, 66(2), 223–231.CrossRefGoogle Scholar
  77. 77.
    Wang, L. X. (1999). Analysis and design of hierarchical fuzzy systems. IEEE Transactions on Fuzzy Systems, 7(5), 617–624.CrossRefGoogle Scholar
  78. 78.
    Wang, L. X., & Mendel, J. M. (1992). Generating fuzzy rules by learning from examples. IEEE Transactions on Systems, Man, and Cybernetics, 22(6), 1414–1427.MathSciNetCrossRefGoogle Scholar
  79. 79.
    Wang, L. X., & Mendel, J. M. (1992). Fuzzy basis functions, universal approximation, and orthogonal least-squares learning. IEEE Transactions on Neural Networks, 3(5), 807–814.CrossRefGoogle Scholar
  80. 80.
    Wang, L. X., & Wei, C. (2000). Approximation accuracy of some neuro-fuzzy approaches. IEEE Transactions on Fuzzy Systems, 8(4), 470–478.CrossRefGoogle Scholar
  81. 81.
    Wu, S., & Er, M. J. (2000). Dynamic fuzzy neural networks—A novel approach to function approximation. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 30(2), 358–364.CrossRefGoogle Scholar
  82. 82.
    Yan, H.-S., & Xu, D. (2007). An approach to estimating product design time based on fuzzy \(\nu \)-support vector machine. IEEE Transactions on Neural Networks, 18(3), 721–731.MathSciNetCrossRefGoogle Scholar
  83. 83.
    Zhang, D., Bai, X. L., & Cai, K. Y. (2004). Extended neuro-fuzzy models of multilayer perceptrons. Fuzzy Sets and Systems, 142, 221–242.MathSciNetzbMATHCrossRefGoogle Scholar
  84. 84.
    Zhou, S.-M., & Gan, J. Q. (2007). Constructing L2-SVM-based fuzzy classifiers in high-dimensional space with automatic model selection and fuzzy rule ranking. IEEE Transactions on Fuzzy Systems, 15(3), 398–409.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringConcordia UniversityMontrealCanada
  2. 2.Xonlink Inc.HangzhouChina

Personalised recommendations