Skip to main content
SpringerLink
Log in
Book cover

International Conference on Innovative Techniques and Applications of Artificial Intelligence

SGAI 2019: Artificial Intelligence XXXVI pp 3–17Cite as

CascadeML: An Automatic Neural Network Architecture Evolution and Training Algorithm for Multi-label Classification (Best Technical Paper)

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11927))

Abstract

In multi-label classification a datapoint can be labelled with more than one class at the same time. A common but trivial approach to multi-label classification is to train individual binary classifiers per label, but the performance can be improved by considering associations between the labels, and algorithms like classifier chains and RAKEL do this effectively. Like most machine learning algorithms, however, these approaches require accurate hyperparameter tuning, a computationally expensive optimisation problem. Tuning is important to train a good multi-label classifier model. There is a scarcity in the literature of effective multi-label classification approaches that do not require extensive hyperparameter tuning. This paper addresses this scarcity by proposing CascadeML, a multi-label classification approach based on cascade neural network that takes label associations into account and requires minimal hyperparameter tuning. The performance of the CasecadeML approach is evaluated using 10 multi-label datasets and compared with other leading multi-label classification algorithms. Results show that CascadeML performs comparatively with the leading approaches but without a need for hyperparameter tuning.

This research was supported by Science Foundation Ireland (SFI) under Grant Number SFI/12/RC/2289_P2.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    A version of CascadeML is available at: https://github.com/phoxis/CascadeML.

References

  1. Baluja, S., Fahlman, S.: Reducing network depth in the cascade-correlation learning architecture. Technical report CMU-CS-94-209, Carnegie Mellon University, Pittsburgh, PA, October 1994

    Google Scholar 

  2. Charte, F., Rivera, A., del Jesus, M.J., Herrera, F.: Concurrence among Imbalanced labels and its influence on multilabel resampling algorithms. In: Polycarpou, M., de Carvalho, A.C.P.L.F., Pan, J.-S., Woźniak, M., Quintian, H., Corchado, E. (eds.) HAIS 2014. LNCS (LNAI), vol. 8480, pp. 110–121. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07617-1_10

    Chapter  Google Scholar 

  3. Chen, Z., Chi, Z., Fu, H., Feng, D.: Multi-instance multi-label image classification: a neural approach. Neurocomputing 99, 298–306 (2013)

    Article  Google Scholar 

  4. Cheng, W., Hullermeier, E.: Combining instance-based learning and logistic regression for multilabel classification. Mach. Learn. 76(2–3), 211–225 (2009)

    Article  Google Scholar 

  5. Crammer, K., Singer, Y.: A family of additive online algorithms for category ranking. J. Mach. Learn. Res. 3, 1025–1058 (2003)

    MathSciNet  MATH  Google Scholar 

  6. Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems 2, pp. 524–532. Morgan-Kaufmann (1990)

    Google Scholar 

  7. Feurer, M., Klein, A., Eggensperger, K., Springenberg, J., Blum, M., Hutter, F.: Efficient and robust automated machine learning. In: Advances in Neural Information Processing Systems 28, pp. 2962–2970 (2015)

    Google Scholar 

  8. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  9. Grodzicki, R., Mańdziuk, J., Wang, L.: Improved multilabel classification with neural networks. In: Rudolph, G., Jansen, T., Beume, N., Lucas, S., Poloni, C. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 409–416. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87700-4_41

    Chapter  Google Scholar 

  10. Hansen, L.K., Pedersen, M.W.: Controlled growth of cascade correlation nets. In: Marinaro, M., Morasso, P.G. (eds.) ICANN 1994, pp. 797–800. Springer, London (1994). https://doi.org/10.1007/978-1-4471-2097-1_189

    Chapter  Google Scholar 

  11. Herrera, F., Charte, F., Rivera, A.J., del Jesús, M.J.: Multilabel Classification - Problem Analysis, Metrics and Techniques. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41111-8

    Book  Google Scholar 

  12. Igel, C., Hüsken, M.: Improving the Rprop learning algorithm. In: Proceedings of the Second International ICSC Symposium on Neural Computation (NC 2000), vol. 2000, pp. 115–121. Citeseer (2000)

    Google Scholar 

  13. Kelleher, J.D., Mac Namee, B., D’Arcy, A.: Fundamentals of Machine Learning for Predictive Data Analytics: Algorithms, Worked Examples, and Case Studies. The MIT Press, Cambridge (2015)

    MATH  Google Scholar 

  14. Madjarov, G., Kocev, D., Gjorgjevikj, D., Dz̆eroski, S.: An extensive experimental comparison of methods for multi-label learning. Pattern Recogn. 45(9), 3084–3104 (2012)

    Google Scholar 

  15. Mencia, E.L., Furnkranz, J.: Pairwise learning of multilabel classifications with perceptrons. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 2899–2906, June 2008

    Google Scholar 

  16. Nam, J., Kim, J., Loza Mencía, E., Gurevych, I., Fürnkranz, J.: Large-scale multi-label text classification — revisiting neural networks. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014. LNCS (LNAI), vol. 8725, pp. 437–452. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44851-9_28

    Chapter  Google Scholar 

  17. Nissen, S.: Large scale reinforcement learning using q-sarsa (\(\lambda \)) and cascading neural networks. Unpublished masters thesis, Department of Computer Science, University of Copenhagen, København, Denmark (2007)

    Google Scholar 

  18. Pakrashi, A., Greene, D., Mac Namee, B.: Benchmarking multi-label classification algorithms. In: 24th Irish Conference on Artificial Intelligence and Cognitive Science (AICS 2016). CEUR Workshop Proceedings (2016)

    Google Scholar 

  19. Phatak, D.S., Koren, I.: Connectivity and performance tradeoffs in the cascade correlation learning architecture. IEEE Trans. Neural Netw. 5(6), 930–935 (1994)

    Article  Google Scholar 

  20. Prechelt, L.: Investigation of the cascor family of learning algorithms. Neural Netw. 10(5), 885–896 (1997)

    Article  Google Scholar 

  21. Read, J., Pérez-Cruz, F.: Deep learning for multi-label classification. CoRR abs/1502.05988 (2015)

    Google Scholar 

  22. Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Mach. Learn. 85(3), 333–359 (2011)

    Article  MathSciNet  Google Scholar 

  23. Rojas, R.: Neural Networks: A Systematic Introduction. Springer, Heidelberg (1996). https://doi.org/10.1007/978-3-642-61068-4

    Book  MATH  Google Scholar 

  24. de Sá, A.G.C., Freitas, A.A., Pappa, G.L.: Automated selection and configuration of multi-label classification algorithms with grammar-based genetic programming. In: Auger, A., Fonseca, C.M., Lourenço, N., Machado, P., Paquete, L., Whitley, D. (eds.) PPSN 2018. LNCS, vol. 11102, pp. 308–320. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99259-4_25

    Chapter  Google Scholar 

  25. de Sá, A.G.C., Pappa, G.L., Freitas, A.A.: Towards a method for automatically selecting and configuring multi-label classification algorithms. In: GECCO (2017)

    Google Scholar 

  26. Spyromitros, E., Tsoumakas, G., Vlahavas, I.: An empirical study of lazy multilabel classification algorithms. In: Darzentas, J., Vouros, G.A., Vosinakis, S., Arnellos, A. (eds.) SETN 2008. LNCS (LNAI), vol. 5138, pp. 401–406. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87881-0_40

    Chapter  Google Scholar 

  27. Tsoumakas, G., Katakis, I., Vlahavas, I.: Effective and efficient multilabel classification in domains with large number of labels. In: Proceedings of ECML/PKDD 2008 Workshop on Mining Multidimensional Data (MMD 2008), vol. 21, pp. 53–59. sn (2008)

    Google Scholar 

  28. Tsoumakas, G., Spyromitros-Xioufis, E., Vilcek, J., Vlahavas, I.: Mulan: a Java library for multi-label learning. J. Mach. Learn. Res. 12, 2411–2414 (2011)

    MathSciNet  MATH  Google Scholar 

  29. Tsoumakas, G., Vlahavas, I.P.: Random k-labelsets: an ensemble method for multilabel classification. In: ECML (2007)

    Google Scholar 

  30. Waugh, S., Adams, A.: Connection strategies in cascade-correlation. In: The Fifth Australian Conference on Neural Networks, pp. 1–4 (1994)

    Google Scholar 

  31. Wei, Y., et al.: CNN: single-label to multi-label. CoRR abs/1406.5726 (2014)

    Google Scholar 

  32. Wever, M., Mohr, F., Hüllermeier, E.: Automated multi-label classification based on ML-Plan. CoRR abs/1811.04060 (2018)

    Google Scholar 

  33. Yu, Q., Wang, J., Zhang, S., Gong, Y., Zhao, J.: Combining local and global hypotheses in deep neural network for multi-label image classification. Neurocomputing 235, 38–45 (2017)

    Article  Google Scholar 

  34. Zhang, M.L., Zhou, Z.H.: ML-kNN: a lazy learning approach to multi-label learning. Pattern Recogn. 40, 2038–2048 (2007)

    Article  Google Scholar 

  35. Zhang, M.L.: ML-RBF: RBF neural networks for multi-label learning. Neural Process. Lett. 29(2), 61–74 (2009)

    Article  Google Scholar 

  36. Zhang, M.L., Zhou, Z.H.: Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans. Knowl. Data Eng. 18(10), 1338–1351 (2006)

    Article  Google Scholar 

  37. Zhang, M.L., Zhou, Z.H.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)

    Article  Google Scholar 

  38. Zhu, J., Liao, S., Lei, Z., Li, S.Z.: Multi-label convolutional neural network based pedestrian attribute classification. Image Vis. Comput. 58, 224–229 (2017)

    Article  Google Scholar 

  39. Zhuang, N., Yan, Y., Chen, S., Wang, H., Shen, C.: Multi-label learning based deep transfer neural network for facial attribute classification. Pattern Recogn. 80, 225–240 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Insight Centre for Data Analytics, University College Dublin, Dublin, Ireland

    Arjun Pakrashi & Brian Mac Namee

Authors
  1. Arjun Pakrashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brian Mac Namee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arjun Pakrashi .

Editor information

Editors and Affiliations

  1. University of Portsmouth, Portsmouth, UK

    Max Bramer

  2. Middlesex University, London, UK

    Miltos Petridis

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pakrashi, A., Mac Namee, B. (2019). CascadeML: An Automatic Neural Network Architecture Evolution and Training Algorithm for Multi-label Classification (Best Technical Paper). In: Bramer, M., Petridis, M. (eds) Artificial Intelligence XXXVI. SGAI 2019. Lecture Notes in Computer Science(), vol 11927. Springer, Cham. https://doi.org/10.1007/978-3-030-34885-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-34885-4_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-34884-7

  • Online ISBN: 978-3-030-34885-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation