Abstract
Evidential calibration methods of binary classifiers improve upon probabilistic calibration methods by representing explicitly the calibration uncertainty due to the amount of training (labelled) data. This justified yet undesirable uncertainty can be reduced by adding training data, which are in general costly. Hence the need for strategies that, given a pool of unlabelled data, will point to interesting data to be labelled, i.e., to data inducing a drop in uncertainty greater than a random selection. Two such strategies are considered in this paper and applied to an ensemble of binary SVM classifiers on some classical binary classification datasets. Experimental results show the interest of the approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
\(Bel^\varTheta _x\) must be induced by a source [7]. It may be obtained by a number of evidential methods to statistical inference, and in particular the likelihood-based evidential method [8] in which case \(Bel^\varTheta _x\) is the consonant belief function whose contour function is the normalized likelihood function given the observed data x.
- 2.
References
Niculescu-Mizil, A., Caruana, R.: Predicting good probabilities with supervised learning. In: Proceedings of ICML, pp. 625–632 (2005)
Zhong, W., Kwok, J.T.: Accurate probability calibration for multiple classifiers. In: Proceedings of IJCAI, pp. 1939–1945 (2013)
Xu, P., Davoine, F., Zha, H., Denoeux, T.: Evidential calibration of binary SVM classifiers. Int. J. Approx. Reason. 72, 55–70 (2016)
Minary, P., Pichon, F., Mercier, D., Lefevre, E., Droit, B.: Evidential joint calibration of binary SVM classifiers using logistic regression. In: Proceedings of SUM, pp. 405–411 (2017)
Settles, B.: Active learning literature survey. Computer Sciences Technical report 1648, University of Wisconsin-Madison (2009)
Kanjanatarakul, O., Sriboonchitta, S., Denœux, T.: Forecasting using belief functions: an application to marketing econometrics. Int. J. Approx. Reason. 55(5), 1113–1128 (2014)
Kanjanatarakul, O., Denœux, T., Sriboonchitta, S.: Prediction of future observations using belief functions: a likelihood-based approach. Int. J. Approx. Reason. 72, 71–94 (2016)
Denoeux, T.: Likelihood-based belief function: Justification and some extensions to low-quality data. Int. J. Approx. Reason. 55(7), 1535–1547 (2014)
Klir, G.J.: Uncertainty and Information: Foundations of Generalized Information Theory. Wiley, Hoboken (2005)
Smets, P., Kennes, R.: The transferable belief model. Artif. Intell. 66(2), 191–234 (1994)
Reineking, T.: Active classification using belief functions and information gain maximization. Int. J. Approx. Reason. 72, 43–54 (2016)
Teugels, J.L.: Some representation of the multivariate Bernoulli and binomial distributions. J. Multivar. Anal. 32, 256–268 (1990)
Acknowledgements
This work is funded in part by the ELSAT2020 project, which is co-financed by the European Union with the European Regional Development Fund, the French state and the Hauts de France Region Council.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Ramel, S., Pichon, F., Delmotte, F. (2018). Active Evidential Calibration of Binary SVM Classifiers. In: Destercke, S., Denoeux, T., Cuzzolin, F., Martin, A. (eds) Belief Functions: Theory and Applications. BELIEF 2018. Lecture Notes in Computer Science(), vol 11069. Springer, Cham. https://doi.org/10.1007/978-3-319-99383-6_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-99383-6_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-99382-9
Online ISBN: 978-3-319-99383-6
eBook Packages: Computer ScienceComputer Science (R0)