Automatic Recurrent and Feed-Forward ANN Rule and Expression Extraction with Genetic Programming

  • Julian Dorado
  • Juan R. Rabuñal
  • Antonino Santos
  • Alejandro Pazos
  • Daniel Rivero
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2439)


Various rule-extraction techniques using ANN have been used so far, most of them being applied on multi-layer ANN, since they are more easily handled. In many cases, extraction methods focusing on different types of networks and training have been implemented. However, there are virtually no methods that view the extraction of rules from ANN as systems which are independent from their architecture, training and internal distribution of weights, connections and activation functions. This paper proposes a ruleextraction system of ANN regardless of their architecture (multi-layer or recurrent), using Genetic Programming as a rule-exploration technique.


Artificial Neural Network Genetic Programming Fuzzy Rule Rule Extraction Symbolic Regression 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tickle, A. B.; Andrews, R.; Golea, M.; Diederich, J. “The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks”. IEEE Transaction on Neural Networks, vol. 9 n° 6, pp 1057–1068, 1998.CrossRefGoogle Scholar
  2. 2.
    Cramer, N. L. “A Representation for the Adaptive Generation of Simple Sequential Programs”, Grefenstette: Proc. of 1st International Conference on GA, 1985.Google Scholar
  3. 3.
    Fujiki C. “Using the Genetic Algorithm to Generate Lisp Source Code to Solve the Prisoner’s Dilemma”, International Conf on GAs, pp. 236–240, 1987.Google Scholar
  4. 4.
    Friedberg R. “A learning machine: Part I”, IBM Journal, pp. 2–13, 1958.Google Scholar
  5. 5.
    Friedberg R. M., Dunham B., North J. H. “A learning machine: Part II”, IBM Journal of Research and Development, 3(3) 282–287, 1959.MathSciNetCrossRefGoogle Scholar
  6. 6.
    Koza J. “Genetic Programming. On the Programming of Computers by means of Natural Selection”. The Mit Press, Cambridge, Massachusetts, 1992.zbMATHGoogle Scholar
  7. 7.
    Koza J. “Genetic Programming II: Automatic Discovery of Reusable Programs”. The Mit Press, Cambridge, Massachusetts, 1994.zbMATHGoogle Scholar
  8. 8.
    Koza J., Forrest H., Andre D., Keane M. “Genetic Programming III: Darwinian Invention and Problem Solving”. Morgan Kaufmann Publishers, San Francisco, 1999.zbMATHGoogle Scholar
  9. 9.
    Fayyad U., Piatetsky-Shapiro G., Smyth P., Uthurusamy R.: “Advances in Knowledge Discovery and Data Mining”. AAAI/MIT Press, 1996Google Scholar
  10. 10.
    Bonarini A.: "Evolutionary Learning of Fuzzy Rules: Competition and Cooperation", Fuzzy Modelling: Paradigms and Practice, W. Pedrycz (Ed.), Kluwer Academic Press, 1996Google Scholar
  11. 11.
    Jang J., Sun C. “Functional equivalence between radial basis function networks and fuzzy inference systems”. IEEE Transactions on Neural Networks, vol. 4, pp 156–158, 1992.CrossRefGoogle Scholar
  12. 12.
    Buckley J. J., Hayashi Y., Czogala E. “On the equivalence of neural nets and fuzzy expert systems”, Fuzzy Sets Systems, N° 53, pp 129–134, 1993.Google Scholar
  13. 13.
    Benítez J. M., Castro J. L., Requena I. “ Are artificial neural networks black boxes? ” IEEE Transactions on Neural Networks, vol. 8, n° 5, pp 1156–1164, 1997.CrossRefGoogle Scholar
  14. 14.
    Andrews R. Diederich J., Tickle A. “A Survey and Critique of Techniques For Extracting Rules From Trained ANN”. Knowledge Based Systems 8, pp 373–389, 1995.CrossRefGoogle Scholar
  15. 15.
    Andrews R., Cable R., Diederich J., et al: “An evaluation and comparison of techniques for extracting and refining rules from ANN”, QUT NRC Technical report, 1996.Google Scholar
  16. 16.
    Towell G., Shavlik J. W. “Knowledge-Based ANN”. AI, 70, pp 119–165, 1994.zbMATHGoogle Scholar
  17. 17.
    Thrun S. “Extracting rules from networks with distributed representations”. NIPS, G. Tesauro, D. Touretzky, T. Leen (eds), MIT Press, 1995.Google Scholar
  18. 18.
    Pop E., Hayward R., Diederich J. “RULENEG: Extracting Rules from a Trained ANN by Stepwise Negation”. Queensland University of Technology, Neurocomputing Research Centre. QUT NRC Technical report, 1994.Google Scholar
  19. 19.
    Tickle A. B., Orlowski M., Diedrich J. “DEDEC: A methodology for extracting rules from trained artificial neural networks”. Queensland Univ. of Technology, Neurocomputing Research Centre. Technical report, 1996.Google Scholar
  20. 20.
    Chalup S., Hayward R., Diedrich J. “Rule extraction from artificial neural networks trained on elementary number classification task”. Queensland University of Technology, Neurocomputing Research Centre. QUT NRC Technical report, 1998.Google Scholar
  21. 21.
    Visser U., Tickle A., Hayward R., Andrews R. “Rule-Extraction from trained neural networks: Different techniques for the determination of herbicides for the plant protection advisory system PRO_PLANT”. Proc. of the rule extraction from trained ANN workshop, Brighton, UK, pp 133–139. 1996.Google Scholar
  22. 22.
    Keedwell E., Narayanan A., Savic D. “Creating rules from trained neural networks using genetic algorithms”. IJCSS, vol. 1, N° 1, pp 30–42. 2000.Google Scholar
  23. 23.
    Wong M. L., Leung K. S.: “Data Mining using Grammar Based Genetic Programming and Applications”, Kluwer Academic Publishers, 2000.Google Scholar
  24. 24.
    Duch W., Adamczak R., Grabczewski K.: “A new methodology of extraction, optimisation and application of crisp and fuzzy logical rules”, IEEE Trans. on N.N., vol. 11, n° 2, 2000.Google Scholar
  25. 25.
    Montana D. J.: “Strongly Typed Genetic Programming”, Evolutionary Computation, The MIT Press, pp. 199–200, Cambridge, MA, 1995.Google Scholar
  26. 26.
    Mertz C., Murphy P.: UCI repository of machine learning databases,
  27. 27.
    Ster B., Dobnikar A.: “Neural networks in medical diagnosis: Compararison with other methods”, A. Bulsari et al. eds, Proc. Int. Conf. EANN’96, pp. 427–430, 1996.Google Scholar
  28. 28.
    Jankowski N., Kadirkamanathan V.: “Statistical Control of RBF-like Networks for Classification”, 7th International Conf. on ANN, pp. 385–390, Lausanne, Switzerland, 1997.Google Scholar
  29. 29.
    Shang N., Breiman L.: “Distribution based trees are more accurate”, Int. Conf. On Neural Information Processing, vol. 1, pp. 133–138, Hong Kong, 1996.Google Scholar
  30. 30.
    Mackey M., Glass L.: “Oscillation and chaos in physiological control systems”, Science, pp. 197–287, 1977.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Julian Dorado
    • 1
  • Juan R. Rabuñal
    • 1
  • Antonino Santos
    • 1
  • Alejandro Pazos
    • 1
  • Daniel Rivero
    • 1
  1. 1.Facultad InformáticaUniv. da CoruñaA CoruñaSpain

Personalised recommendations