Skip to main content
SpringerLink
Log in

Recursive Rule Extraction from NN using Reverse Engineering Technique

  • Research Paper
  • Published:
New Generation Computing Aims and scope Submit manuscript

Abstract

This paper proposes an algorithm named Reverse Engineering Recursive Rule Extraction (RE-Re-RX) for symbolic rule extraction from neural network with mixed attributes. RE-Re-RX algorithm is an extension of the existing Recursive Rule Extraction (Re-RX) algorithm. Re-RX algorithm generates disjoint rules for continuous and discrete attributes. The algorithm first generates rules for discrete attributes. A rule for discrete attributes is further refined recursively if it does not produce satisfactory result. A rule is refined by generating rules with the discrete attributes (if present) that are not covered by the rule or else the process is terminated by generating rules with the continuous attributes (if present). The novelty of the proposed RE-Re-RX algorithm lies in generating rules for continuous attributes. Re-RX generates linear hyper plane for continuous attributes which may not be able to deal with the non-linearity present in data. To overcome this limitation RE-Re-RX algorithm generates simple rules for continuous attributes in the form of input data ranges and target. RE-Re-RX uses the concept of Rule Extraction by Reverse Engineering the NN (RxREN) algorithm in slightly different way to generate rules. RxREN only uses misclassified patterns, whereas RE-Re-RX uses both classified and misclassified patterns of each continuous attribute to calculate input data ranges for constructing rules. The proposed algorithm is validated with six benchmark datasets. The experimental results clearly show the superiority of the proposed algorithm to Re-RX.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Malone, J., McGarry, K., Wermter, S., Bowerman, C.: Data mining using rule extraction from Kohonen self-organising maps. Neural Comput. Appl. 15(1), 9–17 (2006)

    Article  Google Scholar 

  2. Han, J., Kamber, M., Pei, J.: Data mining: concepts and techniques, 3rd edn. Morgan Kaufmann Publishers, San Francisco (2011)

    MATH  Google Scholar 

  3. Augasta, M.G., Kathirvalavakumar, T.: Reverse engineering the NNs for rule extraction in classification problems. Neural Process. Lett. 35(2), 131–150 (2012)

    Article  Google Scholar 

  4. Setiono, R.: Extracting M-of-N rules from trained NNs. IEEE Trans. Neural Netw. 11(2), 512–519 (2000)

    Article  Google Scholar 

  5. Jivani, K., Ambasana, J., Kanani, S.: A survey on rule extraction approaches based techniques for data classification using NN. Int. J. Futuristic Trends Eng. Technol. 1(1), 4–7 (2014)

    Google Scholar 

  6. Setiono, R., Baesens, B., Mues, C.: Recursive NN rule extraction for data with mixed attributes. IEEE Trans. Neural Netw. 19(2), 299–307 (2008)

    Article  Google Scholar 

  7. Craven, M.W., Shavlik, J.W.: Using NNs for data mining. Future Gen. Comput. Syst. 13(2–3), 21l–229 (1997)

    Google Scholar 

  8. Fu, L.M.: Rule generation from neural networks. IEEE Trans. Syst. Man Cybern. 28(8), 1114–1124 (1994)

    Google Scholar 

  9. Towell, G., Shavlik, J.: The extraction of refined rules from knowledge based NNs. Mach. Learn. 13(1), 71–101 (1993)

    Google Scholar 

  10. Setiono, R., Liu, H.: Symbolic representation of NNs. IEEE Comput. 29(3), 71–77 (1996)

    Article  Google Scholar 

  11. Setiono, R., Liu, H.: NeuroLinear: from NNs to oblique decision rules. Neurocomputing 17, 1–24 (1997)

    Article  Google Scholar 

  12. Taha, I.A., Ghosh, J.: Symbolic interpretation of artificial NNs. IEEE Trans. Knowl. Data Eng. 11(3), 448–463 (1999)

    Article  Google Scholar 

  13. Anbananthen, S.K., Sainarayanan, G., Chekima, A., Teo, J.: Data mining using pruned artificial NN tree (ANNT). Inf. Commun. Technol. 1, 1350–1356 (2006)

    Google Scholar 

  14. Khan, I., Kulkarni, A.: Knowledge extraction from survey data using neural networks. Procedia Comput. Sci. 20, 433–438 (2013)

    Article  Google Scholar 

  15. Odajimaa, K., Hayashi, Y., Tianxia, G., Setiono, R.: Greedy rule generation from discrete data and its use in NN rule extraction. NNs 21(7), 1020–1028 (2008)

    MATH  Google Scholar 

  16. Wang, J.G., Yang, J.H., Zhang, W.X., Xu, J.W.: Rule extraction from artificial NN with optimized activation functions. In: IEEE 3rd International Conference on Intelligent System and Knowledge Engineering, pp. 873–879. (2008)

  17. Hara, A., Hayashi, Y.: Ensemble NN rule extraction using Re-RX algorithm. NNs (IJCNN) 1–6 (2012)

  18. Hayashi, Y., Sato, R., Mitra, S.: A new approach to three ensemble NN rule extraction using Recursive-Rule Extraction algorithm. NNs (IJCNN) 1–7 (2013)

  19. Hruschka, E.R., Ebecken, N.F.F.: Extracting rules from multilayer perceptrons in classification problems: a clustering-based approach. Neurocomputing 70(1–3), 384–397 (2006)

    Article  Google Scholar 

  20. Kahramanli, H., Allahverdi, N.: Rule extraction from trained adaptive NNs using artificial immune systems. Expert Syst. Appl. 36(2), 1513–1522 (2009)

    Article  Google Scholar 

  21. Setiono, R., Azcarraga, A., Hayashi, Y.: MofN rule extraction from neural networks trained with augmented discretized input. In: International Joint Conference on Neural Networks (IJCNN) 6–11 July, Beijing, China, pp. 1079–1086. (2014). https://doi.org/10.1109/ijcnn.2014.6889691

  22. Sestito, S., Dillon, T.: Automated knowledge acquisition of rules with continuously valued attributes. In: Proceedings of 12th International Conference on Expert Systems and their Applications, pp. 645–656. (1992)

  23. Craven, M., Shavlik, J.: Extracting tree-structured representations of trained network. Adv. Neural Inf. Process. Syst. (NIPS) 8, 24–30 (1996)

    Google Scholar 

  24. Setiono, R.: Extracting rules from NNs by pruning and hidden-unit splitting. Neural Comput. 9(1), 205–225 (1997)

    Article  MATH  Google Scholar 

  25. Liu, H., Tan, S.T.: X2R: A fast rule generator. In: Proceedings of the 7th IEEE International Conference on Systems, Man and Cybernetics, vol. 2, pp. 631–635. (1995)

  26. Etchells, T.A., Lisboa, P.J.G.: Orthogonal search-based rule extraction (OSRE) for trained NNs: a practical and efficient approach. IEEE Trans. NNs 17(2), 374–384 (2006)

    Article  Google Scholar 

  27. Biswas, S.K., Chakraborty, M., Purkayastha, B., Thounaojam, D.M., Roy, P.: Rule extraction from training data using neural network. Int. J. Artif. Intell. Tool 26, 3 (2017)

    Google Scholar 

  28. de Fortuny, E.J., Martens, D.: Active learning-based pedagogical rule extraction. IEEE Trans. NNs Learn. Syst. 26(11), 2664–2677 (2015)

    Article  MathSciNet  Google Scholar 

  29. Rudy, S., Kheng, W.: FERNN: an algorithm for fast extraction of rules from NNs. Appl. Intell. 12(1), 15–25 (2000)

    Google Scholar 

  30. Iqbal, R.A.: Eclectic rule extraction from NNs using aggregated decision trees. In: IEEE, 7th International Conference on Electrical & Computer Engineering (ICECE), pp. 129–132 (2012)

  31. Hayashi, Y., Nakano, S.: Use of a recursive-rule extraction algorithm with j48graft to archive highly accurate and concise rule extraction from a large breast cancer dataset. Inform. Med. Unlocked 1, 9–16 (2016)

    Article  Google Scholar 

  32. Hayashi, Y., Yukita, S.: Rule extraction using recursive-rule extraction algorithm with J48graft combined with sampling selection techniques for the diagnosis of type 2 diabetes mellitus in the Pima Indian dataset. Inform. Med. Unlocked 2, 92–104 (2016)

    Article  Google Scholar 

  33. Hayashi, Y., Nakano, S., Fujisawa, S.: Use of the recursive-rule extraction algorithm with continuous attributes to improve diagnostic accuracy in thyroid disease. Inform. Med. Unlocked. 1, 1–8 (2015)

    Article  Google Scholar 

  34. Hayashi, Y.: Application of a rule extraction algorithm family based on the Re-RX algorithm to financial credit risk assessment from a Pareto optimal perspective. Oper. Res. Perspect. 3, 32–42 (2016)

    Article  MathSciNet  Google Scholar 

  35. Setiono, R.: A penalty-function approach for pruning feedforward NNs. Neural Comput. 9(1), 185–204 (1997)

    Article  MATH  Google Scholar 

  36. Permanasari, A.E., Rambli, D.R.A., Dominic, P.D.D.: Forecasting of salmonellosis incidence in human using artificial NN (ANN). In: Computer and Automation Engineering (ICCAE), the 2nd International Conference, vol. 1, pp. 136–139. (2010)

  37. Mondal, S.C., Mandal, P.: Application of artificial NN for modeling surface roughness in centerless grinding operation. In: 5th International & 26th All India Manufacturing Technology, Design and Research Conference, IIT Guwahati, Assam, India, 12–14 December 2014

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, National Institute of Technology Silchar, Silchar, Assam, 788010, India

    Manomita Chakraborty, Saroj Kr. Biswas & Biswajit Purkayastha

Authors
  1. Manomita Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saroj Kr. Biswas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Biswajit Purkayastha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manomita Chakraborty.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chakraborty, M., Biswas, S.K. & Purkayastha, B. Recursive Rule Extraction from NN using Reverse Engineering Technique. New Gener. Comput. 36, 119–142 (2018). https://doi.org/10.1007/s00354-018-0031-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00354-018-0031-9

Keywords

Search

Navigation