Abstract
The extraction of logical rules from data by means of artificial neural networks is receiving increasingly much attention. The meaning the extracted rules may convey is primarily determined by the set of their possible truth values, according to which two basic kinds of rules can be differentiated - Boolean and fuzzy. Though a wide spectrum of theoretical principles has been proposed for ANN-based rule extraction, most of the existing methods still rely mainly on heuristics. Moreover, so far apparently no particular principles have been employed for the extraction of both kinds of rules, what can be a serious drawback when switching between Boolean and fuzzy rules. This paper presents a mathematically well founded approach based on piecewise-linear activation functions, which is suitable for the extraction of both kinds of rules. Basic properties of piecewise-linear neural networks are reviewed, most importantly, the replaceability of suboptimal computable mappings, and the preservation of polyhedra. Based on those results, a complete algorithm for the extraction of Boolean rules with that approach is given. In addition, two modifications of the algorithm are described, relying on different assumptions about the way how the properties of a polyhedron determine the decision to replace the polyhedron with a hyperrectangle. Finally, a biological application in which the presented approach has been successfully employed is briefly sketched.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
S. Aguzzoli and D. Mundici. Weierstrass approximations by Lukasiewicz formulas with one quantified variable. In 31st IEEE International Symposium on Multiple-Valued Logic, 2001.
R. Andrews, J. Diederich, and A.B. Tickle. Survey and critique of techniques for extracting rules from trained artificical neural networks. Knowledge Based Systems, 8:378–389, 1995.
L.O. Cignoli, I.M.L. D'Ottaviano, and D. Mundici. Algebraic Foundations of Many-valued Reasoning. Kluwer Academic Publishers, Dordrecht, 2000.
J.E. Dennis and R.B. Schnabel. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice Hall, Englewood Cliffs, 1983.
W. Duch, R. Adamczak, and K. Grabczewski. Extraction of logical rules from neural networks. Neural Processing Letters, 7:211–219, 1998.
W. Duch, R. Adamczak, and K. Grabczewski. A new methodology of extraction, optimization and application of crisp and fuzzy logical rules. IEEE Transactions on Neural Networks, 11:277–306, 2000.
G.D. Finn. Learning fuzzy rules from data. Neural Computing & Applications, 8:9–24, 1999.
M.T. Hagan, H.B. Demuth, and M.H. Beale. Neural Network Design. PWS Publishing, Boston, 1996.
M.T. Hagan and M. Menhaj. Training feedforward networks with the Marquadt algorithm. IEEE Transactions on Neural Networks, 5:989–993, 1994.
P. Hájek. Metamathematics of Fuzzy Logic. Kluwer Academic Publishers, Dordrecht, 1998.
P. Hájek and T. Havránek. Mechanizing Hypothesis Formation. SpringerVerlag, Berlin, 1978.
M. Holeňa. Ordering of neural network architectures. Neural Network World, 3:131–160, 1993.
M. Holeňa. Lattices of neural network architectures. Neural Network World, 4:435–464, 1994.
M. Holeňa. Observational logic integrates data mining based on statistics and neural networks. In D.A. Zighed, J. Komorowski, and J.M. Żytkov, editors, Principles of Data Mining and Knowledge Discovery, pages 440–445. Springer Verlag, Berlin, 2000
M. Holeňa. Mining rules from empirical data with an ecological application. Technical report, Brandenburg University of Technology, Cottbus, 2002. ISBN 3-934934-07-2, 62 pages.
K. Hornik, M. Stinchcombe, H. White, and P. Auer. Degree of approximation results for feedforward networks approximating unknown mappings and their derivatives. Neural Computation, 6:1262–1275, 1994.
P. Howes and N. Crook. Using input parameter influences to support the decisions of feedforward neural networks. Neurocomputing, 24:191–206, 1999.
M. Ishikawa. Rule extraction by successive regularization. Neural Networks, 13:1171–1183, 2000.
V. Kůrková. Kolmogorov’s theorem and multilayer neural networks. Neural Networks, 5:501–506, 1992.
M. Leshno, V.Y. Lin, A. Pinkus, and S. Schocken. Multilayer feedforward networks with a non-polynomial activation can approximate any function. Neural Networks, 6:861–867, 1993.
W. Maass. Bounds for the computational power and learning complexity of analog neural nets. SIAM Journal on Computing, 26:708–732, 1997.
F. Maire. Rule-extraction by backpropagation of polyhedra. Neural Networks, 12:717–725, 1999.
E.N. Mayoraz. On the complexity of recognizing regions computable by two-layered perceptrons. Annals of Mathematics and Artificial Intelligence, 24:129–153, 1998.
S. Mitra, R.K. De, and S.K. Pal. Knowledge-based fuzzy MLP for classification and rule generation. IEEE Transactions on Neural Networks, 8:1338–1350, 1997.
S. Mitra and Y. Hayashi. Neuro-fuzzy rule generation: Survey in soft computing framework. IEEE Transactions on Neural Networks, 11:748–768, 2000.
D. Nauck, U. Nauck, and R. Kruse. Generating classification rules with the neuro-fuzzy system NEFCLASS. In Proceedings of the Biennial Conference of the North American Fuzzy Information Processing Society NAFIPS’96, pages 466–470, 1996.
D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations by error backpropagation. In D.E. Rumelhart and J.L. McClelland, editors, Parallel Distributed Processing: Experiments in the Microstructure of Cognition, pages 318–362, 1986.
A.B. Tickle, R. Andrews, M. Golea, and J. Diederich. The truth will come to light: Directions and challenges in extracting rules from trained artificial neural networks. IEEE Transactions on Neural Networks, 9:1057–1068, 1998.
H. Tsukimoto. Extracting rules from trained neural networks. IEEE Transactions on Neural Networks, 11:333–389, 2000.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Holeňa, M. (2002). Extraction of Logical Rules from Data by Means of Piecewise-Linear Neural Networks. In: Lange, S., Satoh, K., Smith, C.H. (eds) Discovery Science. DS 2002. Lecture Notes in Computer Science, vol 2534. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36182-0_18
Download citation
DOI: https://doi.org/10.1007/3-540-36182-0_18
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-00188-1
Online ISBN: 978-3-540-36182-4
eBook Packages: Springer Book Archive