Abstract
In the last decade, the learning from label proportions problem has attracted the attention of the machine learning community. Many learning methodologies have been proposed, although the evaluation with real label proportions data has hardly been explored. This paper proposes an adaptation of the area under the precision-recall curve metric to the problem of learning from label proportions. The actual performance is bounded by minimum and maximum approximations. Additionally, an approximate estimation which takes advantage of low-uncertain bags is proposed. The benefits of this proposal are illustrated by means of an empirical study.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2011). http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/
Chen, S., Liu, B., Qian, M., Zhang, C.: Kernel k-means based framework for aggregate outputs classification. In: IEEE ICDM Workshops, pp. 356–361 (2009)
Davis, J., Goadrich, M.: The relationship between precision-recall and ROC curves. In: Proceedings of 23rd ICML, pp. 233–240 (2006)
Dery, L.M., Nachman, B., Rubbo, F., Schwartzman, A.: Weakly supervised classification in high energy physics. J. High Energy Phys. 2017(5), 145 (2017)
Fish, B., Reyzin, L.: On the complexity of learning from label proportions. In: Proceedings of 26th IJCAI, pp. 19–25 (2017)
Frank, A., Asuncion, A.: UCI machine learning repository (2010). http://archive.ics.uci.edu/ml
Hernández-González, J., Inza, I., Crisol-Ortiz, L., Guembe, M.A., Iñarra, M.J., Lozano, J.A.: Fitting the data from embryo implantation prediction: learning from label proportions. Stat. Methods Med. Res. 27(4), 1056–1066 (2018)
Hernández-González, J., Inza, I., Lozano, J.A.: Learning Bayesian network classifiers from label proportions. Pattern Recognit. 46(12), 3425–3440 (2013)
Hernández-González, J., Inza, I., Lozano, J.A.: Weak supervision and other non-standard classification problems: a taxonomy. Pattern Recognit. Lett. 69, 49–55 (2016)
Hübner, D., Verhoeven, T., Schmid, K., Müller, K.R., Tangermann, M., Kindermans, P.J.: Learning from label proportions in brain-computer interfaces: online unsupervised learning with guarantees. PLoS One 12(4), e0175856 (2017)
Kück, H., de Freitas, N.: Learning about individuals from group statistics. In: Proceedings of 21st UAI, pp. 332–339 (2005)
Mitchell, T.: Machine Learning. McGraw Hill, New York City (1997)
Musicant, D.R., Christensen, J.M., Olson, J.F.: Supervised learning by training on aggregate outputs. In: Proceedings of 7th IEEE ICDM, pp. 252–261 (2007)
Patrini, G., Nock, R., Rivera, P., Caetano, T.: (Almost) no label no cry. In: Proceedings of NIPS 27, pp. 190–198 (2014)
Pérez-Ortiz, M., Gutiérrez, P.A., Carbonero-Ruz, M., Hervás-Martínez, C.: Adapting linear discriminant analysis to the paradigm of learning from label proportions. In: Proceedings of IEEE SSCI, pp. 1–7 (2016)
Qi, Z., Meng, F., Tian, Y., Niu, L., Shi, Y., Zhang, P.: Adaboost-LLP: a boosting method for learning with label proportions. IEEE Trans. Neural Netw. Learn. Syst., 1–12 (2017, early access)
Quadrianto, N., Smola, A.J., Caetano, T.S., Le, Q.V.: Estimating labels from label proportions. J. Mach. Learn. Res. 10, 2349–2374 (2009)
Rüping, S.: SVM classifier estimation from group probabilities. In: Proceedings of 27th ICML, pp. 911–918 (2010)
Stolpe, M., Morik, K.: Learning from label proportions by optimizing cluster model selection. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011. LNCS (LNAI), vol. 6913, pp. 349–364. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23808-6_23
Sun, T., Sheldon, D., O’Connor, B.: A probabilistic approach for learning with label proportions applied to the US presidential election. In: IEEE ICDM (2017)
Yu, F.X., Liu, D., Kumar, S., Jebara, T., Chang, S.: \(\propto \)SVM for learning with label proportions. In: Proceedings of 30th ICML, pp. 504–512 (2013)
Acknowledgments
This work has been partially supported by the Basque Government (IT609-13, Elkartek BID3A), and the Spanish Ministry of Economy and Competitiveness (TIN2016-78365-R).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Hernández-González, J. (2018). Evaluation in Learning from Label Proportions: An Approximation to the Precision-Recall Curve. In: Herrera, F., et al. Advances in Artificial Intelligence. CAEPIA 2018. Lecture Notes in Computer Science(), vol 11160. Springer, Cham. https://doi.org/10.1007/978-3-030-00374-6_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-00374-6_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00373-9
Online ISBN: 978-3-030-00374-6
eBook Packages: Computer ScienceComputer Science (R0)