Abstract
The previous chapters pretended that performance evaluation in machine learning is a fairly straightforward matter. All it takes is to apply the induced classifier to a set of examples whose classes are known, and then count the number of errors the classifier has made. In reality, things are not as simple. Error rate rarely paints the whole picture, and there are situations in which it can even be misleading. This is why the conscientious engineer wants to be acquainted with other criteria to assess the classifiers’ performance. This knowledge will enable her to choose the one that is best in capturing the behavioral aspects of interest.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The concrete techniques employed to this end are somewhat more advanced than those discussed in this textbook, and are thus not treated here.
- 2.
Some computation will be necessary if we decide to remove noisy or redundant examples.
References
Dietterich, T. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10, 1895–1923.
Hellman, M. E. (1970). The nearest neighbor classification rule with the reject option. IEEE Transactions on Systems Science and Cybernetics, 6, 179–185.
Kubat, M., Holte, R., & Matwin, S. (1997). Learning when negatives examples abound. In Proceedings of the European conference on machine learning (ECML’97), Apr 1997, Prague (pp. 146–153).
Louizou, G. & Maybank, S. J. (1987). The nearest neighbor and the bayes error rates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9, 254–262.
Wolpert, D. (1996). The lack of a priori distinctions between learning algorithms. Neural Computation, 8, 1341–1390.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Kubat, M. (2017). Performance Evaluation. In: An Introduction to Machine Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-63913-0_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-63913-0_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-63912-3
Online ISBN: 978-3-319-63913-0
eBook Packages: Computer ScienceComputer Science (R0)