Abstract
In this paper, the problem of implementing the reject option in support vector machines (SVMs) is addressed. We started by observing that methods proposed so far simply apply a reject threshold to the outputs of a trained SVM. We then showed that, under the framework of the structural risk minimisation principle, the rejection region must be determined during the training phase of a classifier. By applying this concept, and by following Vapnik’s approach, we developed a maximum margin classifier with reject option. This led us to a SVM whose rejection region is determined during the training phase, that is, a SVM with embedded reject option. To implement such a SVM, we devised a novel formulation of the SVM training problem and developed a specific algorithm to solve it. Preliminary results on a character recognition problem show the advantages of the proposed SVM in terms of the achievable error-reject trade-off.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chow, C.K.: An optimum character recognition system using decision functions. IRE Trans. on Electronic Computers 6 (1957) 247–254
Chow, C.K.: On optimum error and reject tradeoff. IEEE Trans. on Information Theory 16 (1970) 41–46
Bishop, CM.: Neural Networks for Pattern Recognition. Oxford University Press (1995)
Ruck, D.W., Rogers, S.K., Kabrisky, M., Oxley, M., Suter, B.: The multilayer perceptron as an approximation to a Bayes optimal discriminant function. IEEE Trans, on Neural Networks 4 (1990) 296–298
Mukherjee, S., Tamayo, P., Slonim, D., Verri, A., Golub, T., Mesirov, J.P., Poggio, T.: Support vector machine classification of microarray data. Tech. report. Massachusetts Institute of Thechnology (1998)
Duin, R.P.W., Tax, D.M.J.: Classifier conditional posterior probabilities. In: Amin, A., Dori, D., Pudil, P., Freeman, H. (eds.): Advances in Pattern Recognition. Lecture Notes in Computer Science 1451, Springer, Berlin (1998) 611–619
Platt, J.C.: Probabilistic outputs for support vector machines and comparison to regularised likelihood methods. In.: Smola, A.J., Bartlett, P., Schölkopf, B., Schuurmans, D. (eds.): Advances in Large Margin Classifiers. Mit Press (1999)
Madevska-Bogdanova, A., Nikolic, D.: A new approach on modifying SVM outputs. Proc. of the IEEE-INNS-ENNS Int. Joint Conference on Neural Networks, Vol. 6 (2000) 395–398
Hastie, T., Tibshirani, R.: Classification by pairwise coupling. Technical Report. Stanford University and University of Toronto (1996)
Kwok, J.T.-Y.: Moderating the outputs of support vector machines. IEEE Transactions on Neural Networks 10(1999) 1018–1031
Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)
Vapnik, V.N.: Estimation of Dependencies Based on Empirical Data, Addendum 1. Springer-Verlag, New York (1982)
Cortes, C, Vapnik, V.N.: Support vector networks. Machine Learning 20 (1995) 1–25
Bazaraa, M.S., Sherali, H.D., Shetty, CM. Nonlinear Programming. Theory and Algorithms. Wiley (1992)
Platt, J.C.: Fast training of supprt vector machines using sequential minimal optimisation. In: B. Schölkopf, Burges, C.J.C., Smola, A.J. (eds.): Advances in Kernel Methods — Support Vector Learning. MIT Press (1999)
Fumera, G.: Advanced Methods for Pattern Recognition with Reject Option. Ph.D. Thesis. University of Cagliari (2002)
Basu, M., Ho, T.K. The learning behavior of single neuron classifiers on linearly separable or nonseparable input. Proc. of the 1999 Int. Joint Conference on Neural Networks. Washington, DC (1999)
Joachims, T.: Making large-scale SVM learning practical. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.): Advances in Kernel Methods — Support Vector Learning. MIT Press (1999) 169–184
Fumera, G., Roli, F., Giacinto, G.: Reject option with multiple thresholds. Pattern Recognition 33 (2000) 165–167
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fumera, G., Roli, F. (2002). Support Vector Machines with Embedded Reject Option. In: Lee, SW., Verri, A. (eds) Pattern Recognition with Support Vector Machines. SVM 2002. Lecture Notes in Computer Science, vol 2388. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45665-1_6
Download citation
DOI: https://doi.org/10.1007/3-540-45665-1_6
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44016-1
Online ISBN: 978-3-540-45665-0
eBook Packages: Springer Book Archive