Skip to main content
SpringerLink
Log in

Rule Extraction from Neural Networks and Support Vector Machines for Credit Scoring

  • Chapter
  • First Online:
Data Mining: Foundations and Intelligent Paradigms

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 25))

Abstract

In this chapter we describe how comprehensible rules can be extracted from artificial neural networks (ANN) and support vector machines (SVM). ANN and SVM are two very popular techniques for pattern classification. In the business intelligence application domain of credit scoring, they have been shown to be effective tools for distinguishing between good credit risks and bad credit risks. The accuracy obtained by these two techniques is often higher than that from decision tree methods. Unlike decision tree methods, however, the classifications made by ANN and SVM are difficult to understand by the end-users as outputs from ANN and SVM are computed as nonlinear mapping of the input data attributes. We describe two rule extraction methods that we have developed to overcome this difficulty. These rule extraction methods enable the users to obtain comprehensible propositional rules from ANN and SVM. Such rules can be easily verified by the domain experts and would lead to a better understanding about the data in hand.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Andrews, R., Diederich, J., Tickle, A.B.: A survey and critique of techniques for extracting rules from trained neural networks. Knowledge Based Systems 8(6), 373–389 (1995)

    Article  Google Scholar 

  2. Baesens, B., Van Gestel, T., Viaene, S., Stepanova, M., Suykens, J., Vanthienen, J.: Benchmarking state-of-the-art classification algorithms for credit scoring. Journal of the Operational Research Society 54(6), 627–635 (2003)

    Article  MATH  Google Scholar 

  3. Baesens, B., Setiono, R., Mues, C., Vanthienen, J.: Using neural network rule extraction and decision tables for credit risk evaluation. Management Science 49(3), 312–329 (2003)

    Article  Google Scholar 

  4. Baesens, B., Van Gestel, T., Viaene, S., Stepanova, M., Suykens, J., Vanthienen, J.: Benchmarking state-of-the-art classification algorithms for credit scoring. Journal of the Operational Research Society 54(6), 627–635 (2003)

    Article  MATH  Google Scholar 

  5. Barakat, N., Diederich, J.: Eclectic rule extraction from support vector machines. International Journal of Computational Intelligence 2(1), 59–62 (2005)

    Google Scholar 

  6. Barakat, N.H., Bradley, A.P.: Rule extraction from support vector machines: Measuring the explanation capability using the area under the ROC curve. In: Proc. of ICPR, vol. (2), pp. 812–815. IEEE Computer Society, Los Alamitos (2006)

    Google Scholar 

  7. Barakat, N.H., Bradley, A.P.: Rule extraction from support vector machines: A sequential covering approach. IEEE Transactions on Knowledge and Data Engineering 19(6), 729–741 (2007)

    Article  Google Scholar 

  8. Battiti, R.: First- and second-order methods for learning: Between steepest descent and Newton’s method. Neural Computation 4, 141–166 (1992)

    Article  Google Scholar 

  9. Bishop, C.M.: Neural networks for pattern recognition. Oxford University Press, Oxford (1995)

    Google Scholar 

  10. Cohen, W.W.: Fast effective rule induction. In: Proc. of the 12th International Conference on Machine Learning, pp. 115–123 (1995)

    Google Scholar 

  11. Cohn, D., Atlas, L., Ladner, R.: Improving generalization with active learning. Machine Learning 15(2), 201–221 (1994)

    Google Scholar 

  12. Cristianini, N., Shawe-Taylor, J.: An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, New York (2000)

    Google Scholar 

  13. Dennis Jr., J.E., Schnabel, R.E.: Numerical methods for unconstrained optimization and nonlinear equations. Prentice Halls, Englewood Cliffs (1983)

    MATH  Google Scholar 

  14. Downs, T., Gates, K.E., Masters, A.: Exact Simplification of support vector solutions. Journal of Machine Learning Research 2, 293–297 (2001)

    Google Scholar 

  15. Fawcett, T.: PRIE: A system for generating rulelists to maximize ROC performance. Data Mining and Knowledge Discovery 17(2), 207–224 (2008)

    Article  MathSciNet  Google Scholar 

  16. Fung, G., Sandilya, S., Rao, R.B.: Rule Extraction from linear support vector machines. In: Proc. 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, pp. 32–40 (2005)

    Google Scholar 

  17. Hertz, J., Krogh, A., Palmer, R.G.: Introduction to the theory of neural computation. Addison-Wesley, Redwood City (1991)

    Google Scholar 

  18. Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366 (1989)

    Article  Google Scholar 

  19. Hsu, C.-W., Lin, C.-J.: A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks 13, 415–425 (2002)

    Article  Google Scholar 

  20. Huysmans, J., Baesens, B., Vanthienen, J.: ITER: An algorithm for predictive regression rule extraction. In: Tjoa, A.M., Trujillo, J. (eds.) DaWaK 2006. LNCS, vol. 4081, pp. 270–279. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  21. Lessmann, S., Baesens, B., Mues, C., Pietsch, S.: Benchmarking classification models for software defect prediction: A proposed framework and novel findings. IEEE Transactions Software Engineering 34(4), 485–496 (2008)

    Article  Google Scholar 

  22. Martens, D., Van Gestel, T., Baesens, B.: Decompositional rule extraction from support vector machines by active learning. IEEE Transactions on Knowledge and Data Engineering 21(2), 178–191 (2009)

    Article  Google Scholar 

  23. Nùñez, H., Angulo, C., Català, A.: Rule extraction from support vector machines. In: Proc. European Symposium on Artificial Neural Networks (ESANN), pp. 107–112 (2002)

    Google Scholar 

  24. Prechelt, L.: PROBEN1 - A set of benchmarks and benchmarking rules for neural network training algorithms. Technical Report 21/94, Fakultät für Informatik. Universität Karlsruhe, Germany. Anonymous ftp, ftp://pub/papers/techreports/1994/1994021.ps.gz on ftp.ira.uka.de

    Google Scholar 

  25. Quinlan, R.: C4.5: Programs for machine learning. Morgan Kaufman, San Mateo (1993)

    Google Scholar 

  26. Saar-Tsechansky, M., Provost, F.: Decision-centric active learning of binary-outcome models. Information Systems Research 18(1), 4–22 (2007)

    Article  Google Scholar 

  27. Setiono, R.: A neural network construction algorithm which maximizes the likelihood function. Connection Science 7(2), 147–166 (1995)

    Article  Google Scholar 

  28. Setiono, R., Hui, L.C.K.: Use of quasi-Newton method in a feedforward neural network construction algorithm. IEEE Transactions on Neural Networks 6(2), 326–332 (1995)

    Google Scholar 

  29. Setiono, R.: A penalty function approach for pruning feedforward neural networks. Neural Computation 9(1), 185–204 (1997)

    Article  MATH  Google Scholar 

  30. Setiono, R., Baesens, B., Mues, C.: Recursive neural network rule extraction for data with mixed attributes. IEEE Transactions on Neural Networks 19(2), 299–307 (2008)

    Article  Google Scholar 

  31. Sexton, R.S., McMurtrey, S., Cleavenger, D.J.: Knowledge discovery using a neural network simultaneous optimization algorithm on a real world classification problem. European Journal of Operational Research 168, 1009–1018 (2006)

    Article  MATH  Google Scholar 

  32. Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least squares support vector machines. World Scientific, Singapore (2003)

    Google Scholar 

  33. Thomas, L., Edelman, D., Crook, J.: Credit scoring and its applications. SIAM, Philadelphia (2002)

    Book  MATH  Google Scholar 

  34. Tickle, A.B., Andrews, R., Golea, M., Diederich, J.: The truth will come to light: Directions and challenges in extracting the knowledge embedded within trained artificial neural networks. IEEE Transactions on Neural Networks 9(6), 1057–1068 (1998)

    Article  Google Scholar 

  35. Tipping, M.: Sparse bayesian learning and the relevance vector machine. Journal of Machine Learning Research 1, 211–244 (2001)

    MATH  MathSciNet  Google Scholar 

  36. Vapnik, V.N.: The nature of statistical learning theory. Springer, New York (1995)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, National University of Singapore, 13 Computing Drive, Singapore, Singapore, 117417

    Rudy Setiono

  2. Department of Decision Sciences and Information Management, Katholieke Universiteit Leuven, Naamsestraat 69, B-3000, Leuven, Belgium

    Bart Baesens & David Martens

  3. Department of Business Administration and Public Management, Hogeschool Ghent, University Ghent, Voskenslaan, Ghent, Belgium

    David Martens

Authors
  1. Rudy Setiono
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bart Baesens
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Martens
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Letters & Science, Dept. Statistics & Applied Probability, University of California, Santa Barbara, South Hall 5504, Santa Barbara, 93106-3110, California, USA

    Dawn E. Holmes

  2. , School of Electrical and Information Eng, University of South Australia, Adelaide,, Mawson Lakes Campus, SA 5095, Australia

    Lakhmi C Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Setiono, R., Baesens, B., Martens, D. (2012). Rule Extraction from Neural Networks and Support Vector Machines for Credit Scoring. In: Holmes, D., Jain, L. (eds) Data Mining: Foundations and Intelligent Paradigms. Intelligent Systems Reference Library, vol 25. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23151-3_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-23151-3_13

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-23150-6

  • Online ISBN: 978-3-642-23151-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation