Heuristic Classifier Chains for Multi-label Classification

  • Tomasz Kajdanowicz
  • Przemyslaw Kazienko
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8132)


Multi-label classification, in opposite to conventional classification, assumes that each data instance may be associated with more than one labels simultaneously. Multi-label learning methods take advantage of dependencies between labels, but this implies greater learning computational complexity.

The paper considers Classifier Chain multi-label classification method, which in original form is fast, but assumes the order of labels in the chain. This leads to propagation of inference errors down the chain. On the other hand recent Bayes-optimal method, Probabilistic Classifier Chain, overcomes this drawback, but is computationally intractable. In order to find the trade off solution it is presented a novel heuristic approach for finding appropriate label order in chain. It is demonstrated that the method obtains competitive overall accuracy and is also tractable to higher-dimensional data.


Binary Relevance Label Space Label Relevance Label Dependency Inference Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kajdanowicz, T., Kazienko, P.: Hybrid repayment prediction for debt portfolio. In: Nguyen, N.T., Kowalczyk, R., Chen, S.-M. (eds.) ICCCI 2009. LNCS, vol. 5796, pp. 850–857. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  2. 2.
    Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Machine Learning 85(3), 333–359 (2011)CrossRefGoogle Scholar
  3. 3.
    Wolpert, D.H.: Stacked generalization. Neural Networks 5, 241–259 (1992)CrossRefGoogle Scholar
  4. 4.
    Ting, K.M., Witten, I.H.: Issues in stacked generalization. Journal of Artificial Intelligence Research 10, 271–289 (1999)zbMATHGoogle Scholar
  5. 5.
    Tsoumakas, G., Dimou, A., Spyromitros, E., Mezaris, V., Kompatsiaris, I., Vlahavas, I.: Correlation-based pruning of stacked binary relevance models for multi-label learning. In: Proceedings of 1st International Workshop on Learning from Multi-Label Data, MLD 2009, at the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, pp. 101–116 (2009)Google Scholar
  6. 6.
    Dembczynski, K., Cheng, W., Hullermeier, E.: Bayes optimal multilabel classification via probabilistic classifier chains. In: Proceedings of the 27th International Conference on Machine Learning, ICML 2010, Haifa, Israel, pp. 279–286. Omnipress (June 2010)Google Scholar
  7. 7.
    Dembczynski, K., Waegeman, W., Cheng, W., Hullermeier, E.: On label dependence in multi-label classification. In: Workshop Proceedings of Learning from Multi-Label Data, Haifa, Israel, pp. 5–12 (June 2010)Google Scholar
  8. 8.
    Kajdanowicz, T., Kazienko, P.: Structured output element ordering in boosting-based classification. In: Corchado, E., Kurzyński, M., Woźniak, M. (eds.) HAIS 2011, Part II. LNCS, vol. 6679, pp. 221–228. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  9. 9.
    Kajdanowicz, T., Kazienko, P.: Learning and inference order in structured output elements classification. In: Pan, J.-S., Chen, S.-M., Nguyen, N.T. (eds.) ACIIDS 2012, Part I. LNCS, vol. 7196, pp. 301–309. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recognition 37(9), 1757–1771 (2004)CrossRefGoogle Scholar
  11. 11.
    Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. In: Advances in Neural Information Processing Systems. Neural Information Processing Systems, pp. 681–687. MIT Press (2001)Google Scholar
  12. 12.
    Trohidis, K., Tsoumakas, G., Kalliris, G., Vlahavas, I.: Multilabel classification of music into emotions. In: Proceedings of 9th International Conference on Music Information Retrieval, ISMIR 2008, Philadelphia, PA, USA, pp. 325–330 (2008)Google Scholar
  13. 13.
    Pestian, J., Brew, C., Matykiewicz, P., Hovermale, D., Johnson, N., Bretonnel Cohen, K., Duch, W.: A shared task involving multi-label classification of clinical free text. In: Proceedings of the Workshop on BioNLP 2007: Biological, Translational, and Clinical Language Processing. Association of Computational Linguistics (2007)Google Scholar
  14. 14.
    Diplaris, S., Tsoumakas, G., Mitkas, P., Vlahavas, I.: Protein classification with multiple algorithms. In: Bozanis, P., Houstis, E.N. (eds.) PCI 2005. LNCS, vol. 3746, pp. 448–456. Springer, Heidelberg (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Tomasz Kajdanowicz
    • 1
  • Przemyslaw Kazienko
    • 1
  1. 1.Faculty of Computer Science and Management, Institute of InformaticsWroclaw University of TechnologyWroclawPoland

Personalised recommendations