Use of Mutual Information to Extract Rules from Artificial Neural Networks
This paper investigates the application of the mutual information for the evaluation of neuron inputs and for the selection of the relevant ones. The rules extraction method is based on the notion of weights templates, parameterizing regions of weights space using the mutual information criteria. The simulation results obtained with this method are very satisfactory.
KeywordsArtificial Neural Network Mutual Information Average Mutual Information Input Weight Rule Extraction
Unable to display preview. Download preview PDF.
- J.A. Alexander and M.C. Mozer. Template-based algorithms for connectionnist rule extraction, pages 609–616. MIT Press, Cambridge, MA, 1995.Google Scholar
- R. Andrews, J. Diederich, and A.B. Tickle. A survey and critique of techniques for extracting rules from trained artificial neural networks. Technical Report QUTNRC-95-01-02, Queensland University of Technology, Brisbane, Australia, 1995.Google Scholar
- B.V. Bonnlander and A.S. Weigned. Selecting input variables using mutual information and non-parametric density estimation. In Proceedings of the 1994 International Symposium on Artificial Neural Networks (ISANN’94). Tainan, Taiwan, 1996.Google Scholar
- K.J. Cherkauer and J.W. Shavlik. Rapid quality estimation of neural network input representations. Neural Information Processing Systems, 8, 1996.Google Scholar
- D.H. Fisher and K.B. McKusick. An empirical comparison of ID3 and back-propagation. In Proceedings of the eleventh International Joint Conference on Artificial Intelligence, pages 788–793, 1989.Google Scholar
- L.M. Fu. Rule generation from neural networks. IEEE Transaction on Systems, Man, and Cybernetics, 28(8):1114–1124, 1994.Google Scholar
- P.M. Murphy and M.J. Pazzani. ID2-of-3: Constructive induction of N-of-M concepts for discrimination in decision trees. In Proceedings of the Eight International Machine Learning Workshop, pages 183–187, 1991.Google Scholar
- J.W. Shavlik and G.G. Towell. Extracting refined rules from knowledge-based neural networks. Machine learning, pages 71–101, 1993.Google Scholar