New Generation Computing

, Volume 36, Issue 2, pp 119–142 | Cite as

Recursive Rule Extraction from NN using Reverse Engineering Technique

  • Manomita Chakraborty
  • Saroj Kr. Biswas
  • Biswajit Purkayastha
Research Paper
  • 84 Downloads

Abstract

This paper proposes an algorithm named Reverse Engineering Recursive Rule Extraction (RE-Re-RX) for symbolic rule extraction from neural network with mixed attributes. RE-Re-RX algorithm is an extension of the existing Recursive Rule Extraction (Re-RX) algorithm. Re-RX algorithm generates disjoint rules for continuous and discrete attributes. The algorithm first generates rules for discrete attributes. A rule for discrete attributes is further refined recursively if it does not produce satisfactory result. A rule is refined by generating rules with the discrete attributes (if present) that are not covered by the rule or else the process is terminated by generating rules with the continuous attributes (if present). The novelty of the proposed RE-Re-RX algorithm lies in generating rules for continuous attributes. Re-RX generates linear hyper plane for continuous attributes which may not be able to deal with the non-linearity present in data. To overcome this limitation RE-Re-RX algorithm generates simple rules for continuous attributes in the form of input data ranges and target. RE-Re-RX uses the concept of Rule Extraction by Reverse Engineering the NN (RxREN) algorithm in slightly different way to generate rules. RxREN only uses misclassified patterns, whereas RE-Re-RX uses both classified and misclassified patterns of each continuous attribute to calculate input data ranges for constructing rules. The proposed algorithm is validated with six benchmark datasets. The experimental results clearly show the superiority of the proposed algorithm to Re-RX.

Keywords

Data mining Neural networks Rule extraction Re-RX algorithm RxREN algorithm Classification 

References

  1. 1.
    Malone, J., McGarry, K., Wermter, S., Bowerman, C.: Data mining using rule extraction from Kohonen self-organising maps. Neural Comput. Appl. 15(1), 9–17 (2006)CrossRefGoogle Scholar
  2. 2.
    Han, J., Kamber, M., Pei, J.: Data mining: concepts and techniques, 3rd edn. Morgan Kaufmann Publishers, San Francisco (2011)MATHGoogle Scholar
  3. 3.
    Augasta, M.G., Kathirvalavakumar, T.: Reverse engineering the NNs for rule extraction in classification problems. Neural Process. Lett. 35(2), 131–150 (2012)CrossRefGoogle Scholar
  4. 4.
    Setiono, R.: Extracting M-of-N rules from trained NNs. IEEE Trans. Neural Netw. 11(2), 512–519 (2000)CrossRefGoogle Scholar
  5. 5.
    Jivani, K., Ambasana, J., Kanani, S.: A survey on rule extraction approaches based techniques for data classification using NN. Int. J. Futuristic Trends Eng. Technol. 1(1), 4–7 (2014)Google Scholar
  6. 6.
    Setiono, R., Baesens, B., Mues, C.: Recursive NN rule extraction for data with mixed attributes. IEEE Trans. Neural Netw. 19(2), 299–307 (2008)CrossRefGoogle Scholar
  7. 7.
    Craven, M.W., Shavlik, J.W.: Using NNs for data mining. Future Gen. Comput. Syst. 13(2–3), 21l–229 (1997)Google Scholar
  8. 8.
    Fu, L.M.: Rule generation from neural networks. IEEE Trans. Syst. Man Cybern. 28(8), 1114–1124 (1994)Google Scholar
  9. 9.
    Towell, G., Shavlik, J.: The extraction of refined rules from knowledge based NNs. Mach. Learn. 13(1), 71–101 (1993)Google Scholar
  10. 10.
    Setiono, R., Liu, H.: Symbolic representation of NNs. IEEE Comput. 29(3), 71–77 (1996)CrossRefGoogle Scholar
  11. 11.
    Setiono, R., Liu, H.: NeuroLinear: from NNs to oblique decision rules. Neurocomputing 17, 1–24 (1997)CrossRefGoogle Scholar
  12. 12.
    Taha, I.A., Ghosh, J.: Symbolic interpretation of artificial NNs. IEEE Trans. Knowl. Data Eng. 11(3), 448–463 (1999)CrossRefGoogle Scholar
  13. 13.
    Anbananthen, S.K., Sainarayanan, G., Chekima, A., Teo, J.: Data mining using pruned artificial NN tree (ANNT). Inf. Commun. Technol. 1, 1350–1356 (2006)Google Scholar
  14. 14.
    Khan, I., Kulkarni, A.: Knowledge extraction from survey data using neural networks. Procedia Comput. Sci. 20, 433–438 (2013)CrossRefGoogle Scholar
  15. 15.
    Odajimaa, K., Hayashi, Y., Tianxia, G., Setiono, R.: Greedy rule generation from discrete data and its use in NN rule extraction. NNs 21(7), 1020–1028 (2008)MATHGoogle Scholar
  16. 16.
    Wang, J.G., Yang, J.H., Zhang, W.X., Xu, J.W.: Rule extraction from artificial NN with optimized activation functions. In: IEEE 3rd International Conference on Intelligent System and Knowledge Engineering, pp. 873–879. (2008)Google Scholar
  17. 17.
    Hara, A., Hayashi, Y.: Ensemble NN rule extraction using Re-RX algorithm. NNs (IJCNN) 1–6 (2012)Google Scholar
  18. 18.
    Hayashi, Y., Sato, R., Mitra, S.: A new approach to three ensemble NN rule extraction using Recursive-Rule Extraction algorithm. NNs (IJCNN) 1–7 (2013)Google Scholar
  19. 19.
    Hruschka, E.R., Ebecken, N.F.F.: Extracting rules from multilayer perceptrons in classification problems: a clustering-based approach. Neurocomputing 70(1–3), 384–397 (2006)CrossRefGoogle Scholar
  20. 20.
    Kahramanli, H., Allahverdi, N.: Rule extraction from trained adaptive NNs using artificial immune systems. Expert Syst. Appl. 36(2), 1513–1522 (2009)CrossRefGoogle Scholar
  21. 21.
    Setiono, R., Azcarraga, A., Hayashi, Y.: MofN rule extraction from neural networks trained with augmented discretized input. In: International Joint Conference on Neural Networks (IJCNN) 6–11 July, Beijing, China, pp. 1079–1086. (2014).  https://doi.org/10.1109/ijcnn.2014.6889691
  22. 22.
    Sestito, S., Dillon, T.: Automated knowledge acquisition of rules with continuously valued attributes. In: Proceedings of 12th International Conference on Expert Systems and their Applications, pp. 645–656. (1992)Google Scholar
  23. 23.
    Craven, M., Shavlik, J.: Extracting tree-structured representations of trained network. Adv. Neural Inf. Process. Syst. (NIPS) 8, 24–30 (1996)Google Scholar
  24. 24.
    Setiono, R.: Extracting rules from NNs by pruning and hidden-unit splitting. Neural Comput. 9(1), 205–225 (1997)CrossRefMATHGoogle Scholar
  25. 25.
    Liu, H., Tan, S.T.: X2R: A fast rule generator. In: Proceedings of the 7th IEEE International Conference on Systems, Man and Cybernetics, vol. 2, pp. 631–635. (1995)Google Scholar
  26. 26.
    Etchells, T.A., Lisboa, P.J.G.: Orthogonal search-based rule extraction (OSRE) for trained NNs: a practical and efficient approach. IEEE Trans. NNs 17(2), 374–384 (2006)CrossRefGoogle Scholar
  27. 27.
    Biswas, S.K., Chakraborty, M., Purkayastha, B., Thounaojam, D.M., Roy, P.: Rule extraction from training data using neural network. Int. J. Artif. Intell. Tool 26, 3 (2017)Google Scholar
  28. 28.
    de Fortuny, E.J., Martens, D.: Active learning-based pedagogical rule extraction. IEEE Trans. NNs Learn. Syst. 26(11), 2664–2677 (2015)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Rudy, S., Kheng, W.: FERNN: an algorithm for fast extraction of rules from NNs. Appl. Intell. 12(1), 15–25 (2000)Google Scholar
  30. 30.
    Iqbal, R.A.: Eclectic rule extraction from NNs using aggregated decision trees. In: IEEE, 7th International Conference on Electrical & Computer Engineering (ICECE), pp. 129–132 (2012)Google Scholar
  31. 31.
    Hayashi, Y., Nakano, S.: Use of a recursive-rule extraction algorithm with j48graft to archive highly accurate and concise rule extraction from a large breast cancer dataset. Inform. Med. Unlocked 1, 9–16 (2016)CrossRefGoogle Scholar
  32. 32.
    Hayashi, Y., Yukita, S.: Rule extraction using recursive-rule extraction algorithm with J48graft combined with sampling selection techniques for the diagnosis of type 2 diabetes mellitus in the Pima Indian dataset. Inform. Med. Unlocked 2, 92–104 (2016)CrossRefGoogle Scholar
  33. 33.
    Hayashi, Y., Nakano, S., Fujisawa, S.: Use of the recursive-rule extraction algorithm with continuous attributes to improve diagnostic accuracy in thyroid disease. Inform. Med. Unlocked. 1, 1–8 (2015)CrossRefGoogle Scholar
  34. 34.
    Hayashi, Y.: Application of a rule extraction algorithm family based on the Re-RX algorithm to financial credit risk assessment from a Pareto optimal perspective. Oper. Res. Perspect. 3, 32–42 (2016)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Setiono, R.: A penalty-function approach for pruning feedforward NNs. Neural Comput. 9(1), 185–204 (1997)CrossRefMATHGoogle Scholar
  36. 36.
    Permanasari, A.E., Rambli, D.R.A., Dominic, P.D.D.: Forecasting of salmonellosis incidence in human using artificial NN (ANN). In: Computer and Automation Engineering (ICCAE), the 2nd International Conference, vol. 1, pp. 136–139. (2010)Google Scholar
  37. 37.
    Mondal, S.C., Mandal, P.: Application of artificial NN for modeling surface roughness in centerless grinding operation. In: 5th International & 26th All India Manufacturing Technology, Design and Research Conference, IIT Guwahati, Assam, India, 12–14 December 2014Google Scholar

Copyright information

© Ohmsha, Ltd. and Springer Japan KK, part of Springer Nature 2018

Authors and Affiliations

  • Manomita Chakraborty
    • 1
  • Saroj Kr. Biswas
    • 1
  • Biswajit Purkayastha
    • 1
  1. 1.Computer Science and Engineering DepartmentNational Institute of Technology SilcharSilcharIndia

Personalised recommendations