MetaUtil: Meta Learning for Utility Maximization in Regression

  • Paula BrancoEmail author
  • Luís Torgo
  • Rita P. Ribeiro
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11198)


Several important real world problems of predictive analytics involve handling different costs of the predictions of the learned models. The research community has developed multiple techniques to deal with these tasks. The utility-based learning framework is a generalization of cost-sensitive tasks that takes into account both costs of errors and benefits of accurate predictions. This framework has important advantages such as allowing to represent more complex settings reflecting the domain knowledge in a more complete and precise way. Most existing work addresses classification tasks with only a few proposals tackling regression problems. In this paper we propose a new method, MetaUtil, for solving utility-based regression problems. The MetaUtil algorithm is versatile allowing the conversion of any out-of-the-box regression algorithm into a utility-based method. We show the advantage of our proposal in a large set of experiments on a diverse set of domains.



This work is partially funded by the ERDF through the COMPETE 2020 Programme within project POCI-01-0145-FEDER-006961, and by National Funds through the FCT as part of project UID/EEA/50014/2013. Paula Branco was supported by a scholarship from the Fundação para a Ciência e Tecnologia (FCT), Portugal (scholarship number PD/BD/105788/2014). The participation of Luis Torgo on this research was undertaken thanks in part to funding from the Canada First Research Excellence Fund for the Ocean Frontier Institute.


  1. 1.
    Ribeiro, R.P.: Utility-based regression. PhD thesis, Department Computer Science, Faculty of Sciences - University of Porto (2011)Google Scholar
  2. 2.
    Elkan, C.: The foundations of cost-sensitive learning. In: IJCAI’01: Proceedings of the 17th International Joint Conference of Artificial Intelligence, vol. 1, pp. 973–978. Morgan Kaufmann Publishers (2001)Google Scholar
  3. 3.
    Torgo, L., Ribeiro, R.: Utility-based regression. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 597–604. Springer, Heidelberg (2007). Scholar
  4. 4.
    Ling, C.X., Sheng, V.S.: Cost-sensitive learning. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning, pp. 231–235. Springer, US, Boston, MA (2011)Google Scholar
  5. 5.
    Domingos, P.: Metacost: a general method for making classifiers cost-sensitive. In: KDD’99: Proceedings of the 5th International Conference on Knowledge Discovery and Data Mining, pp. 155–164. ACM Press (1999)Google Scholar
  6. 6.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley, New York (2012)Google Scholar
  7. 7.
    Bansal, G., Sinha, A.P., Zhao, H.: Tuning data mining methods for cost-sensitive regression: a study in loan charge-off forecasting. J. Manag. Inf. Syst. 25(3), 315–336 (2008)CrossRefGoogle Scholar
  8. 8.
    Zhao, H., Sinha, A.P., Bansal, G.: An extended tuning method for cost-sensitive regression and forecasting. Decis. Support Syst. 51(3), 372–383 (2011)CrossRefGoogle Scholar
  9. 9.
    Hernández-Orallo, J.: Probabilistic reframing for cost-sensitive regression. ACM Trans. Knowl. Discov. Data 8(4), 17:1–17:55 (2014)CrossRefGoogle Scholar
  10. 10.
    Branco, P., Torgo, L., Ribeiro, R.P., Frank, E., Pfahringer, B., Rau, M.M.: Learning through utility optimization in regression tasks. In: 2017 IEEE International Conference on Data Science and Advanced Analytics, DSAA 2017, Tokyo, Japan, pp. 30–39 (2017). Accessed 19–21 Oct 2017Google Scholar
  11. 11.
    Frank, E., Bouckaert, R.R.: Conditional density estimation with class probability estimators. In: Asian Conference on Machine Learning, pp. 65–81. Springer (2009)Google Scholar
  12. 12.
    Rau, M.M., et al.: Accurate photometric redshift probability density estimation-method comparison and application. Mon. Not. R. Astron. Soc. 452(4), 3710–3725 (2015)CrossRefGoogle Scholar
  13. 13.
    Branco, P., Torgo, L., Ribeiro, R.P.: SMOGN: a pre-processing approach for imbalanced regression. In: First International Workshop on Learning with Imbalanced Domains: Theory and Applications, pp. 36–50 (2017)Google Scholar
  14. 14.
    Branco, P., Torgo, L., Ribeiro, R.P.: A survey of predictive modeling on imbalanced domains. ACM Comput. Surv. (CSUR) 49(2), 31 (2016)CrossRefGoogle Scholar
  15. 15.
    Poursabzi-Sangdeh, F., Goldstein, D.G., Hofman, J.M., Vaughan, J.W., Wallach, H.: Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018)
  16. 16.
    Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017)
  17. 17.
    R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2018)Google Scholar
  18. 18.
    Dimitriadou, E., Hornik, K., Leisch, F., Meyer, D., Weingessel, A.: e1071: Misc Functions of the Department of Statistics (e1071), TU Wien (2011)Google Scholar
  19. 19.
    Liaw, A., Wiener, M.: Classification and regression by randomforest. R News 2(3), 18–22 (2002)Google Scholar
  20. 20.
    Domingos, P.: Knowledge acquisition from examples via multiple models. In: Machine Learning - International Workshop Then Conference -, Morgan Kaufmann Publishers, INC., pp. 98–106 (1997)Google Scholar
  21. 21.
    Torgo, L.: An infra-structure for performance estimation and experimental comparison of predictive models in R. In: CoRR arXiv:abs/1412.0436 (2014)
  22. 22.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Paula Branco
    • 1
    • 2
    Email author
  • Luís Torgo
    • 1
    • 2
    • 3
  • Rita P. Ribeiro
    • 1
    • 2
  1. 1.LIAAD - INESC TECPortoPortugal
  2. 2.DCC - Faculdade de CiênciasUniversidade do PortoPortoPortugal
  3. 3.Faculty of Computer ScienceDalhousie UniversityHalifaxCanada

Personalised recommendations