Advertisement

Extremely Randomized CNets for Multi-label Classification

  • Teresa M. A. Basile
  • Nicola Di Mauro
  • Floriana Esposito
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11298)

Abstract

Multi-label classification (MLC) is a challenging task in machine learning consisting in the prediction of multiple labels associated with a single instance. Promising approaches for MLC are those able to capture label dependencies by learning a single probabilistic model—differently from other competitive approaches requiring to learn many models. The model is then exploited to compute the most probable label configuration given the observed attributes. Cutset Networks (CNets) are density estimators leveraging context-specific independencies providing exact inference in polynomial time. The recently introduced Extremely Randomized CNets (XCNets) reduce the structure learning complexity making able to learn ensembles of XCNets outperforming state-of-the-art density estimators. In this paper we employ XCNets for MLC by exploiting efficient Most Probable Explanations (MPE). An experimental evaluation on real-world datasets shows how the proposed approach is competitive w.r.t. other sophisticated methods for MLC.

Keywords

Multi-label classification Cutset networks Tractable probabilistic models 

References

  1. 1.
    Antonucci, A., Corani, G., Mauá, D.D., Gabaglio, S.: An ensemble of Bayesian networks for multilabel classification. In: IJCAI, pp. 1220–1225 (2013)Google Scholar
  2. 2.
    Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recogn. 37(9), 1757–1771 (2004)CrossRefGoogle Scholar
  3. 3.
    Cano, A., Luna, J.M., Gibaja, E.L., Ventura, S.: LAIM discretization for multi-label data. Inf. Sci. 330, 370–384 (2016)CrossRefGoogle Scholar
  4. 4.
    Chow, C., Liu, C.: Approximating discrete probability distributions with dependence trees. IEEE Trans. Inf. Theory 14(3), 462–467 (1968)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Conaty, D., de Campos, C.P., Mauá, D.D.: Approximation complexity of maximum a posteriori inference in sum-product networks. In: UAI (2017)Google Scholar
  6. 6.
    Corani, G., Antonucci, A., Mauá, D.D., Gabaglio, S.: Trading off speed and accuracy in multilabel classification. In: van der Gaag, L.C., Feelders, A.J. (eds.) PGM 2014. LNCS (LNAI), vol. 8754, pp. 145–159. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-11433-0_10CrossRefGoogle Scholar
  7. 7.
    Dembczyński, K., Cheng, W., Hüllermeier, E.: Bayes optimal multilabel classification via probabilistic classifier chains. In: ICML, pp. 279–286 (2010)Google Scholar
  8. 8.
    Dembczyński, K., Waegeman, W., Cheng, W., Hüllermeier, E.: On label dependence and loss minimization in multi-label classification. Mach. Learn. 88(1), 5–45 (2012)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Di Mauro, N., Vergari, A., Basile, T.M.A.: Learning Bayesian random cutset forests. In: Esposito, F., Pivert, O., Hacid, M.-S., Raś, Z.W., Ferilli, S. (eds.) ISMIS 2015. LNCS (LNAI), vol. 9384, pp. 122–132. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-25252-0_13CrossRefGoogle Scholar
  10. 10.
    Di Mauro, N., Vergari, A., Basile, T.M.A., Esposito, F.: Fast and accurate density estimation with extremely randomized cutset networks. In: Ceci, M., Hollmén, J., Todorovski, L., Vens, C., Džeroski, S. (eds.) ECML PKDD 2017. LNCS (LNAI), vol. 10534, pp. 203–219. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-71249-9_13CrossRefGoogle Scholar
  11. 11.
    Di Mauro, N., Vergari, A., Esposito, F.: Learning accurate cutset networks by exploiting decomposability. In: Gavanelli, M., Lamma, E., Riguzzi, F. (eds.) AI*IA 2015. LNCS (LNAI), vol. 9336, pp. 221–232. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24309-2_17CrossRefGoogle Scholar
  12. 12.
    Di Mauro, N., Vergari, A., Esposito, F.: Multi-label classification with cutset networks. In: PGM, vol. 52, pp. 147–158 (2016)Google Scholar
  13. 13.
    Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge (2009)zbMATHGoogle Scholar
  14. 14.
    Llerena, J.V., Mauá, D.D.: On using sum-product networks for multi-label classification. In: BRACIS, pp. 25–30 (2017)Google Scholar
  15. 15.
    Lowd, D., Rooshenas, A.: The libra toolkit for probabilistic models. JMLR 16, 2459–2463 (2015)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Madjarov, G., Kocev, D., Gjorgjevikj, D., Deroski, S.: An extensive experimental comparison of methods for multi-label learning. Pattern Recogn. 45(9), 3084–3104 (2012)CrossRefGoogle Scholar
  17. 17.
    Meila, M., Jordan, M.I.: Learning with mixtures of trees. JMLR 1, 1–48 (2000)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Peharz, R., Gens, R., Pernkopf, F., Domingos, P.: On the latent variable interpretation in sum-product networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(10), 2030–2044 (2017)CrossRefGoogle Scholar
  19. 19.
    Poon, H., Domingos, P.: Sum-product network: a new deep architecture. In: NIPS 2010 Workshop on Deep Learning and Unsupervised Feature Learning (2011)Google Scholar
  20. 20.
    Rahman, T., Gogate, V.: Learning ensembles of cutset networks. In: AAAI (2016)Google Scholar
  21. 21.
    Rahman, T., Kothalkar, P., Gogate, V.: Cutset networks: a simple, tractable, and scalable approach for improving the accuracy of Chow-Liu trees. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014. LNCS (LNAI), vol. 8725, pp. 630–645. Springer, Heidelberg (2014).  https://doi.org/10.1007/978-3-662-44851-9_40CrossRefGoogle Scholar
  22. 22.
    Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds.) ECML PKDD 2009. LNCS (LNAI), vol. 5782, pp. 254–269. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-04174-7_17CrossRefGoogle Scholar
  23. 23.
    Roth, D.: On the hardness of approximate reasoning. Artif. Intell. 82(1–2), 273–302 (1996)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Tsoumakas, G., Katakis, I., Vlahavas, I.: Random k-labelsets for multilabel classification. IEEE Trans. Knowl. Data Eng. 23(7), 1079–1089 (2011)CrossRefGoogle Scholar
  25. 25.
    Tsoumakas, G., Katakis, I., Vlahavas, I.: Mining multi-label data. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, 2nd edn, pp. 667–685. Springer, Boston (2009).  https://doi.org/10.1007/978-0-387-09823-4_34CrossRefGoogle Scholar
  26. 26.
    Vergari, A., Di Mauro, N., Esposito, F.: Simplifying, regularizing and strengthening sum-product network structure learning. In: Appice, A., Rodrigues, P.P., Santos Costa, V., Gama, J., Jorge, A., Soares, C. (eds.) ECML PKDD 2015. LNCS (LNAI), vol. 9285, pp. 343–358. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-23525-7_21CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Teresa M. A. Basile
    • 2
    • 3
  • Nicola Di Mauro
    • 1
  • Floriana Esposito
    • 1
  1. 1.Department of Computer ScienceUniversity of Bari “Aldo Moro”BariItaly
  2. 2.Department of PhysicsUniversity of Bari “Aldo Moro”BariItaly
  3. 3.National Institute for Nuclear Physics (INFN), Bari DivisionBariItaly

Personalised recommendations