Abstract
We present NeuroLinear, a system for extracting oblique decision rules from neural networks that have been trained for classification of patterns. Each condition of an oblique decision rule corresponds to a partition of the attribute space by a hyperplane that is not necessarily axis-parallel. Allowing a set of such hyperplanes to form the boundaries of the decision regions leads to a significant reduction in the number of rules generated while maintaining the accuracy rates of the networks. We describe the components of NeuroLinear in detail using a heart disease diagnosis problem. Our experimental results on real-world datasets show that the system is effective in extracting compact and comprehensible rules with high predictive accuracy from neural networks.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
T.G. Dietterich, H. Hild, and G. Bakiri, “A comparative study of ID3 and back-propagation for English text-to-speech mapping,” in Machine Learning: Proceedings of the Seventh International Conference, Austin, Texas, 1990
D.H. Fisher and K.B. McKusick. “An empirical comparison of ID3 and back-propagation,” in Proceedings of 11th Int. Joint Conf. on AI, pp. 788–793, 1989.
J.R. Quinlan. “Comparing connectionist and symbolic learning methods,” in S.J. Hanson, G.A. Drastall, and R.L. Rivest, eds., Computational Learning Theory and Natural Learning Systems, Vol 1 (A Bradford Book, The MIT Press, 1994) pp. 445–456.
J.W. Shavlik, R.J. Mooney, and G.G. Towell, “Symbolic and neural learning algorithms: An experimental comparison”, Machine Learning, vol. 6, no. 2, 111–143, 1991.
S. Gallant, “Connectionist expert systems,” Comm. of the ACM, vol. 31, no. 2, pp. 152–169, Feb. 1988.
K. Saito and R. Nakano, “Medical diagnosis expert system based on PDP model,” in Proc. IEEE Intl. Conf. on Neural Networks, IEEE Press, New York, pp. 255–266, 1988.
L. Fu, “Rule learning by searching on adapted nets,” in Proc. of the Ninth National Conference on Artificial Intelligence, (1991) pp. 590–595.
G.G. Towell and J.W. Shavlik, “Extracting refined rules from knowledge-based neural networks,” Machine Learning, vol. 13, no. 1, pp. 71–101, 1993.
R. Blassig. “GDS: Gradient descent generation of symbolic classification rules,” in Advances in Neural Information Processing, Vol. 6, (Morgan Kaufmann, Los Angeles CA, 1994) pp. 1093–1100.
R. Setiono and H. Liu, “Symbolic representation of neural networks,” IEEE Computer, March 1996, pp. 71–77.
J.R. Quinlan. C4.5: Programs for Machine Learning, San Mateo, CA: Morgan Kaufmann, 1993.
R. Kerber, “ChiMerge: Discretization of numeric attributes,” in The Proc. of the Ninth National Conference on AI, AAAI Press/The MIT Press, 1992, pp. 123–128.
L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone, Classification and Regression Trees, Wadsworth & Brooks/Cole Advanced Books & Software, 1984.
J. Hertz, A. Krogh, and R.G. Palmer, “Introduction to the theory of neural computation,” Redwood City, CA: Addison Wesley, 1991.
H. Liu and S.T. Tan, “X2R: A fast rule generator,” in Proceedings of IEEE International Conference on Systems, Man and Cybernetics, IEEE Press, 1995.
P.M. Murphy and D.W. Aha, UCI repository of machine learning databases [machine-readable data repository], Department of Information and Computer Science, University of California, Irvine, 1992.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Setiono, R., Liu, H. (1997). NeuroLinear: A system for extracting oblique decision rules from neural networks. In: van Someren, M., Widmer, G. (eds) Machine Learning: ECML-97. ECML 1997. Lecture Notes in Computer Science, vol 1224. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-62858-4_87
Download citation
DOI: https://doi.org/10.1007/3-540-62858-4_87
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-62858-3
Online ISBN: 978-3-540-68708-5
eBook Packages: Springer Book Archive