Abstract
Neural networks offer good generalization performance, noise robustness, and model complexity control. However, neural network mappings are expressed in terms of complicated mathematical functions that are inherently hard to understand. To overcome this limitation rule extraction methods have been proposed. This paper presents a novel method of rule extraction which recursively, in a top-down manner, builds a Reduced Ordered Decision Diagram. The diagram structure allows sharing of nodes, which partially overcomes two problems present in Decision Tree-based rule extraction – the problem of subtree replication and of training set fragmentation. A method for reducing the rule search space by identifying regions in which the network shows similar behavior is presented. Preliminary results of the method performance are reported.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Andersen, H.R.: An introduction to binary decision diagrams. Lect. not., IT University of Copenhagen (1999), http://www.itu.dk/people/hra/notes-index.html
Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems 8(6), 373–389 (1995)
Bryant, R.E.: Graph-based algorithms for boolean function manipulation. IEEE Transactions on Computers 100(35), 677–691 (1986)
Chorowski, J., Zurada, J.: Extracting rules from nns as decision diagrams. IEEE Transactions on Neural Networks (2011), dx.doi.org/10.1109/TNN.2011.2106163
Craven, M.W., Shavlik, J.W.: Using sampling and queries to extract rules from trained neural networks. In: Proceedings of the Eleventh International Conference on Machine Learning, pp. 37–45. Morgan Kaufmann, San Francisco (1994)
Craven, M., Shavlik, J.: Extracting tree-structured representations of trained networks. In: Advances in Neural Information Processing Systems, pp. 24–30. MIT Press, Cambridge (1996)
Etchells, T., Lisboa, P.: Orthogonal search-based rule extraction (osre) for trained neural networks: a practical and efficient approach. IEEE Transactions on Neural Networks 17(2), 374–384 (2006)
Fayyad, U., Irani, K.: Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning, pp. 1022–1027 (1993)
Frank, A., Asuncion, A.: UCI machine learning repository (2010), http://archive.ics.uci.edu/ml
Fu, L.M.: Rule generation from neural networks. IEEE Transactions on Systems, Man and Cybernetics 24(8), 1114–1124 (1994)
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.: The WEKA data mining software: An update. ACM SIGKDD Explorations Newsletter 11(1), 10–18 (2009)
Huysmans, J., Baesens, B., Vanthienen, J.: Using rule extraction to improve the comprehensibility of predictive models. Tech. rep., KU Leuven (2006), http://www.econ.kuleuven.be/fetew/pdf_publicaties/KBI_0612.pdf
Kohavi, R.: Bottom-up induction of oblivious read-once decision graphs: strengths and limitations. In: Proceedings of the Twelfth National Conference on Artificial Intelligence,AAAI 1994, American Association for Artificial Intelligence, Menlo Park, CA, USA, vol. 1, pp. 613–618 (1994)
Kohavi, R., Li, C.H.: Oblivious decision trees graphs and top down pruning. In: Proceedings of the 14th International Joint Conference on Artificial Intelligence, vol. 2, pp. 1071–1077. Morgan Kaufmann Publishers Inc, San Francisco (1995)
Krishnan, R., Sivakumar, G., Bhattacharya, P.: A search technique for rule extraction from trained neural networks. Pattern Recognition Letters 20(3), 273–280 (1999)
Meinel, C., Theobald, T.: Algorithms and Data Structures in VLSI Design, 1st edn. Springer-Verlag New York, Inc., Secaucus (1998)
Mues, C., Baesens, B., Files, C.M., Vanthienen, J.: Decision diagrams in machine learning: an empirical study on real-life credit-risk data. Expert Systems with Applications 27(2), 257–264 (2004)
Oliveira, A.L., Sangiovanni-Vincentelli, A.: Using the minimum description length principle to infer reduced ordered decision graphs. Machine Learning 25, 23–50 (1996) 10.1023/A:1018344122010
Quinlan, J.R.: C4. 5: programs for machine learning. Morgan Kaufmann, San Francisco (1993)
Setiono, R., Baesens, B., Mues, C.: Recursive neural network rule extraction for data with mixed attributes. IEEE Transactions on Neural Networks 19(2), 299–307 (2008)
Setiono, R., Liu, H.: Symbolic representation of neural networks. Computer 29(3), 71–77 (1996)
Thrun, S.B., Bala, J., Bloedorn, E., Bratko, I., Cestnik, B., Cheng, J., Jong, K.D., Dzeroski, S., Fahlman, S.E., Fisher, D., Hamann, R., Kaufman, K., Keller, S., Kononenko, I., Kreuziger, J., Michalski, R., Mitchell, T., Pachowicz, P., Reich, Y., Vafaie, H., Welde, W.V.D., Wenzel, W., Wnek, J., Zhang, J.: The monk’s problems a performance comparison of different learning algorithms. Tech. rep. (1991)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chorowski, J., Zurada, J.M. (2011). Top-Down Induction of Reduced Ordered Decision Diagrams from Neural Networks. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2011. ICANN 2011. Lecture Notes in Computer Science, vol 6792. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21738-8_40
Download citation
DOI: https://doi.org/10.1007/978-3-642-21738-8_40
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21737-1
Online ISBN: 978-3-642-21738-8
eBook Packages: Computer ScienceComputer Science (R0)