Use of Mutual Information to Extract Rules from Artificial Neural Networks

  • T. Nedjari
Conference paper


This paper investigates the application of the mutual information for the evaluation of neuron inputs and for the selection of the relevant ones. The rules extraction method is based on the notion of weights templates, parameterizing regions of weights space using the mutual information criteria. The simulation results obtained with this method are very satisfactory.


Artificial Neural Network Mutual Information Average Mutual Information Input Weight Rule Extraction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    J.A. Alexander and M.C. Mozer. Template-based algorithms for connectionnist rule extraction, pages 609–616. MIT Press, Cambridge, MA, 1995.Google Scholar
  2. [2]
    R. Andrews, J. Diederich, and A.B. Tickle. A survey and critique of techniques for extracting rules from trained artificial neural networks. Technical Report QUTNRC-95-01-02, Queensland University of Technology, Brisbane, Australia, 1995.Google Scholar
  3. [3]
    R. Battiti. Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks, 5(4):537–550, 1994.CrossRefGoogle Scholar
  4. [4]
    B.V. Bonnlander and A.S. Weigned. Selecting input variables using mutual information and non-parametric density estimation. In Proceedings of the 1994 International Symposium on Artificial Neural Networks (ISANN’94). Tainan, Taiwan, 1996.Google Scholar
  5. [5]
    K.J. Cherkauer and J.W. Shavlik. Rapid quality estimation of neural network input representations. Neural Information Processing Systems, 8, 1996.Google Scholar
  6. [6]
    D.H. Fisher and K.B. McKusick. An empirical comparison of ID3 and back-propagation. In Proceedings of the eleventh International Joint Conference on Artificial Intelligence, pages 788–793, 1989.Google Scholar
  7. [7]
    L.M. Fu. Rule generation from neural networks. IEEE Transaction on Systems, Man, and Cybernetics, 28(8):1114–1124, 1994.Google Scholar
  8. [8]
    P.M. Murphy and M.J. Pazzani. ID2-of-3: Constructive induction of N-of-M concepts for discrimination in decision trees. In Proceedings of the Eight International Machine Learning Workshop, pages 183–187, 1991.Google Scholar
  9. [9]
    J.W. Shavlik and G.G. Towell. Extracting refined rules from knowledge-based neural networks. Machine learning, pages 71–101, 1993.Google Scholar

Copyright information

© Springer-Verlag Wien 1998

Authors and Affiliations

  • T. Nedjari
    • 1
  1. 1.LIPN-CNRS URA 1507, Institut GaliléeUniversité Paris 13VilletaneuseFrance

Personalised recommendations