Skip to main content
SpringerLink
Log in
Book cover

Computational Intelligence for Pattern Recognition pp 83–113Cite as

Multi-classifier-Systems: Architectures, Algorithms and Applications

  • Chapter
  • First Online:

Part of the book series: Studies in Computational Intelligence ((SCI,volume 777))

Abstract

In this work multi-classifier-systems (MCS) are discussed. Several fixed and trainable aggregation rules are presented. The most famous examples of MCS, namely bagging and boosting, are explained. Diversity between the base classifiers is a crucial point in order to build accurate MCS. Several criteria to measure diversity in MCS are defined and a motivation for diversity measures, based on the base classifiers’ outputs is given. A case study on pain intensity estimation, based on physiological data streams, is conducted. Within the framework of the case study, different MCS and fusion approaches are evaluated. The case study is conducted on two different data sets, with four and five pain levels respectively, which were induced to the test persons under strictly controlled conditions. The aim of the case study is to implement an automatic pain intensity application system and analyse its effectiveness.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. R.A. Fisher, The use of multiple measurements in taxonomic problems. Ann. Eugenics 7(2), 179–188 (1936)

    Article  Google Scholar 

  2. B.V. Dasarathy, Nearest neighbor (nn) norms: Nn pattern classification techniques. In: IEEE Computer Society Press, Los Alamitos, California (1990)

    Google Scholar 

  3. L. Breiman, J.H. Friedman, R.A. Olshen, C.J. Stone, Classification and Regression Trees. (Wadsworth, 1984)

    Google Scholar 

  4. J.A. Anderson, E. Rosenfeld (eds.), Neurocomputing: Foundations of Research (MIT Press, Cambridge, 1988)

    Google Scholar 

  5. Kittler, J., Roli, F. (eds.): Multiple Classifier Systems, First International Workshop, MCS 2000, Cagliari, Italy, June 21-23, 2000, Proceedings, Lecture Notes in Computer Science, vol. 1857 (Springer, 2000)

    Google Scholar 

  6. Schwenker, F., Roli, F., Kittler, J. (eds.): Multiple Classifier Systems - 12th International Workshop, MCS 2015, Günzburg, Germany, June 29 - July 1, 2015, Proceedings, Lecture Notes in Computer Science, vol. 9132. Springer (2015)

    Google Scholar 

  7. C. Dietrich, F. Schwenker, G. Palm, Classification of time series utilizing temporal and decision fusion, in Multiple Classifier Systems, Second International Workshop, MCS 2001 Cambridge, UK, 2–4 July 2001, Proceedings ed. by J. Kittler, F. Roli. Lecture Notes in Computer Science, vol. 2096 (Springer, 2001), pp. 378–387

    Chapter  Google Scholar 

  8. C. Dietrich, F. Schwenker, G. Palm, Multiple classifier systems for the recognition of orthoptera songs, in Pattern Recognition, 25th DAGM Symposium, Magdeburg, Germany, 10–12 Sept 2003, Proceedings ed. by B. Michaelis, G. Krell. Lecture Notes in Computer Science, vol. 2781 (Springer, 2003), pp. 474–481

    Google Scholar 

  9. C. Thiel, F. Schwenker, G. Palm, Using dempster-shafer theory in MCF systems to reject samples, in Multiple Classifier Systems, 6th International Workshop, MCS 2005, Seaside, CA, USA, 13–15 June 2005, Proceedings ed. by N.C. Oza, R. Polikar, J. Kittler, F. Roli. Lecture Notes in Computer Science, vol. 3541 (Springer, 2005), pp. 118–127

    Chapter  Google Scholar 

  10. M.F.A. Hady, F. Schwenker, Decision templates based RBF network for tree-structured multiple classifier fusion, in Multiple Classifier Systems, 8th International Workshop, MCS 2009, Reykjavik, Iceland, 10–12 June 2009, Proceedings ed. by J.A. Benediktsson, J. Kittler, F. Roli. Lecture Notes in Computer Science, vol. 5519 (Springer, 2009), pp. 92–101

    Google Scholar 

  11. L.I. Kuncheva, C.J. Whitaker, Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51(2), 181–207 (2003)

    Article  Google Scholar 

  12. S. Faußer, F. Schwenker, Neural network ensembles in reinforcement learning. Neural Process. Lett. 41(1), 55–69 (2015)

    Article  Google Scholar 

  13. F. Schwenker, S. Scherer, M. Schmidt, M. Schels, M. Glodek, Multiple classifier systems for the recognition of human emotions, in Multiple Classifier Systems, 9th International Workshop, MCS 2010, Cairo, Egypt, 7–9 Apr 2010, Proceedings ed. by N.E. Gayar, J. Kittler, F. Roli. Lecture Notes in Computer Science, vol. 5997 (Springer, 2010), pp. 315–324

    Chapter  Google Scholar 

  14. M. Schels, F. Schwenker, A multiple classifier system approach for facial expressions in image sequences utilizing GMM supervectors, in 20th International Conference on Pattern Recognition, ICPR 2010 (IEEE Computer Society, Istanbul, Turkey, 2010), pp. 4251–4254

    Google Scholar 

  15. M. Schels, P. Schillinger, F. Schwenker, Training of multiple classifier systems utilizing partially labeled sequential data sets, in ESANN 2011, Proceedings of 19th European Symposium on Artificial Neural Networks, Bruges, Belgium, 27–29 Apr 2011

    Google Scholar 

  16. M. Glodek, S. Tschechne, G. Layher, M. Schels, T. Brosch, S. Scherer, M. Kächele, M. Schmidt, H. Neumann, G. Palm, F. Schwenker, Multiple classifier systems for the classification of audio-visual emotional states, in Affective Computing and Intelligent Interaction - Fourth International Conference, ACII 2011, Memphis, TN, USA, 9–12 Oct 2011, Proceedings ed. by S.K. D’Mello, A.C. Graesser, B.W. Schuller, Martin, J. (eds.), Part II. Lecture Notes in Computer Science, vol. 6975 (Springer, 2011), pp. 359–368

    Chapter  Google Scholar 

  17. M. Glodek, F. Honold, T. Geier, G. Krell, F. Nothdurft, S. Reuter, F. Schüssel, T. Hörnle, K.C.J. Dietmayer, W. Minker, S. Biundo, M. Weber, G. Palm, F. Schwenker, Fusion paradigms in cognitive technical systems for human-computer interaction. Neurocomputing 161, 17–37 (2015)

    Article  Google Scholar 

  18. S. Meudt, D. Zharkov, M. Kächele, F. Schwenker, Multi classifier systems and forward backward feature selection algorithms to classify emotional coloured speech, in 2013 International Conference on Multimodal Interaction, ICMI ’13, ed. by J. Epps, F. Chen, S. Oviatt, K. Mase, A. Sears, K. Jokinen, B.W. Schuller, 9–13 Dec 2013. (ACM, Sydney, NSW, Australia, 2013), pp. 551–556

    Google Scholar 

  19. T.G. Dietterich, Ensemble methods in machine learning, in Multiple Classifier Systems, First International Workshop, MCS 2000, Cagliari, Italy, 21–23 June 2000, Proceedings ed. by J. Kittler, F. Roli. Lecture Notes in Computer Science, vol. 1857 (Springer, 2000). pp. 1–15

    Google Scholar 

  20. R. Polikar, Ensemble based systems in decision making. IEEE Circuits Syst. Mag. 6(3), 21–45 (2006)

    Article  Google Scholar 

  21. L. Xu, A. Krzyzak, C.Y. Suen, Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans. Syst. Man Cybern. 22(3), 418–435 (1992)

    Article  Google Scholar 

  22. L.I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms (Wiley, New Jerssey, 2004)

    Book  Google Scholar 

  23. J. Kittler, M. Hatef, R.P.W. Duin, J. Matas, On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)

    Article  Google Scholar 

  24. D.M.J. Tax, M. van Breukelen, R.P.W. Duin, J. Kittler, Combining multiple classifiers by averaging or by multiplying? Pattern Recognit. 33(9), 1475–1485 (2000)

    Article  Google Scholar 

  25. D.G. Saari, Mathematical structure of voting paradoxes. Econ. Theory 15(1), 55–102 (2000)

    Article  MathSciNet  Google Scholar 

  26. Y.S. Huang, C.Y. Suen, A method of combining multiple experts for the recognition of unconstrained handwritten numerals. IEEE Trans. Pattern Anal. Mach. Intell. 17(1), 90–94 (1995)

    Article  Google Scholar 

  27. L. Kuncheva, J.C. Bezdek, R.P.W. Duin, Decision templates for multiple classifier fusion: an experimental comparison. Pattern Recognit. 34(2), 299–314 (2001)

    Article  Google Scholar 

  28. C. Dietrich, G. Palm, F. Schwenker, Decision templates for the classification of bioacoustic time series. Inf. Fusion 4(2), 101–109 (2003)

    Article  Google Scholar 

  29. R. Penrose, A generalized inverse for matrices. Proc. Camb. Philos. Soc. 51, 406–413 (1955)

    Article  Google Scholar 

  30. F. Schwenker, C.R. Dietrich, C. Thiel, G. Palm, Learning of decision fusion mappings for pattern recognition. Int. J. Artif. Intell. Mach. Learn. (AIML) 6, 17–21 (2006)

    Google Scholar 

  31. G. Giacinto, F. Roli, Design of effective neural network ensembles for image classification purposes. Image Vision Comput. 19(9–10), 699–707 (2001)

    Article  Google Scholar 

  32. R.A. Jacobs, M.I. Jordan, S.J. Nowlan, G.E. Hinton, Adaptive mixtures of local experts. Neural Comput. 3(1), 79–87 (1991)

    Article  Google Scholar 

  33. L. Breiman, Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)

    MATH  Google Scholar 

  34. T.K. Ho, The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20(8), 832–844 (1998)

    Article  Google Scholar 

  35. L. Breiman, Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  36. Y. Freund, R.E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997)

    Article  MathSciNet  Google Scholar 

  37. J. Friedman, T. Hastie, R. Tibshirani, Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors). Ann. Stat. 28(2), 337–407 (2000)

    Article  Google Scholar 

  38. G.I. Webb, Multiboosting: a technique for combining boosting and wagging. Mach. Learn. 40(2), 159–196 (2000)

    Article  Google Scholar 

  39. N.C. Oza, Boosting with averaged weight vectors, in Multiple Classifier Systems, 4th International Workshop, MCS 2003, Guilford, UK, 11–13 June 2003, Proceedings ed. by T. Windeatt, F. Roli. Lecture Notes in Computer Science, vol. 2709 (Springer, 2003), pp. 15–24

    Chapter  Google Scholar 

  40. P.M. Long, SN, V.B.V.: Boosting and microarray data. Mach. Learn. 52(1–2), 31–44 (2003)

    Google Scholar 

  41. G.U. Yule, On the association of attributes in statistics. Phil. Trans. A, 257–319 (1900)

    Google Scholar 

  42. D. Ruta, B. Gabrys, Analysis of the correlation between majority voting error and the diversity measures in multiple classifier systems, in Proceedings of the SOCO/ISFI’2001 Conference, No. 1824-0, ICSC-NAISO (Academic Press, 2001), p. 50

    Google Scholar 

  43. P. Cunningham, J. Carney, Diversity versus quality in classification ensembles based on feature selection, in Machine Learning: ECML 2000, 11th European Conference on Machine Learning, Barcelona, Catalonia, Spain, May 31 - June 2, 2000, Proceedings, ed. by R.L. de Mántaras, E. Plaza, vol. 1810. Lecture Notes in Computer Science (Springer, 2000), pp. 109–116

    Chapter  Google Scholar 

  44. L.K. Hansen, P. Salamon, Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12(10), 993–1001 (1990)

    Article  Google Scholar 

  45. R. Kohavi, D. Wolpert, Bias plus variance decomposition for zero-one loss functions, in Machine Learning, Proceedings of the Thirteenth International Conference (ICML ’96), Bari, Italy, 3–6 July 1996 ed. by L. Saitta (Morgan Kaufmann, 1996), pp. 275–283

    Google Scholar 

  46. D. Partridge, W.J. Krzanowski, Software diversity: practical statistics for its measurement and exploitation. Inf. Softw. Technol. 39(10), 707–717 (1997)

    Article  Google Scholar 

  47. A. Mikami, M. Kudo, A. Nakamura, Diversity measures and margin criteria in multi-class majority vote ensemble, in Multiple Classifier Systems - 12th International Workshop, MCS 2015, Günzburg, Germany, June 29–July 1, 2015, Proceedings ed. by F. Schwenker, F. Roli, J. Kittler. Lecture Notes in Computer Science, vol. 9132 (Springer, 2015), pp. 27–37

    Chapter  Google Scholar 

  48. G. Tsoumakas, I. Partalas, I.P. Vlahavas, An ensemble pruning primer, in Applications of Supervised and Unsupervised Ensemble Methods, Studies in Computational Intelligence, ed. by O. Okun, G. Valentini, vol. 245 (Springer, 2009), pp. 1–13

    Google Scholar 

  49. G. Brown, L.I. Kuncheva, “good” and “bad” diversity in majority vote ensembles, Multiple Classifier Systems, 9th International Workshop, MCS 2010, Cairo, Egypt, 7–9 Apr 2010, Proceedings ed. by N.E. Gayar, J. Kittler. Lecture Notes in Computer Science, vol. 5997 (Springer, 2010), pp. 124–133

    Google Scholar 

  50. S. Walter, S. Gruss, H. Ehleiter, J. Tan, H.C. Traue, S.C. Crawcour, P. Werner, A. Al-Hamadi, A.O. Andrade, The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system, in 2013 IEEE International Conference on Cybernetics, CYBCO 2013, 13–15 June 2013 (IEEE, Lausanne, Switzerland, 2013), pp. 128–131

    Google Scholar 

  51. M. Velana, S. Gruss, G. Layher, P. Thiam, Y. Zhang, D. Schork, V. Kessler, S. Meudt, H. Neumann, J. Kim, F. Schwenker, E. André, H.C. Traue, S. Walter, The senseemotion database: A multimodal database for the development and systematic validation of an automatic pain- and emotion-recognition system, in Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction - 4th IAPR TC 9 Workshop, MPRSS 2016, Cancun, Mexico, December 4, 2016, Revised Selected Papers ed. by F. Schwenker, S. Scherer. Lecture Notes in Computer Science, vol. 10183 (Springer, 2016), pp. 127–139

    Google Scholar 

  52. M. Kächele, P. Werner, A. Al-Hamadi, G. Palm, S. Walter, F. Schwenker, Bio-visual fusion for person-independent recognition of pain intensity, in Multiple Classifier Systems - 12th International Workshop, MCS 2015, Günzburg, Germany, June 29–July 1, 2015, Proceedings ed. by F. Schwenker, F. Roli, J. Kittler. Lecture Notes in Computer Science, vol. 9132 (Springer, 2015), pp. 220–230

    Chapter  Google Scholar 

  53. M. Kächele, P. Thiam, M. Amirian, F. Schwenker, G. Palm, Methods for person-centered continuous pain intensity assessment from bio-physiological channels. J. Sel. Topics Signal Process. 10(5), 854–864 (2016)

    Article  Google Scholar 

  54. V. Kessler, P. Thiam, A. Mohammadreza, F. Schwenker, Pain recognition with camera photoplethysmography, in International Conference on Image Processing Theory, Tools and Applications (2017)

    Google Scholar 

  55. V. Kessler, P. Thiam, M. Amirian, F. Schwenker, Multimodal fusion including camera photoplethysmography for pain recognition, in International Conference on Companion Technology. IEEE International Conference on Companion Technology (2017)

    Google Scholar 

  56. P. Thiam, V. Kessler, F. Schwenker, Hierarchical combination of video features for personalised pain level recognition, in 25th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (2017), pp. 465–470

    Google Scholar 

  57. P. Thiam, V. Kessler, S. Walter, G. Palm, F. Schwenker, Audio-visual recognition of pain intensity. Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction - 4th IAPR TC 9 Workshop, MPRSS 2016, Cancun, Mexico, December 4, 2016, Revised Selected Papers ed. by F. Schwenker, S. Scherer. Lecture Notes in Computer Science, vol. 10183. (Springer, 2016), pp. 110–126

    Google Scholar 

  58. P. Thiam, F. Schwenker, Multi-modal data fusion for pain intensity assessment and classification, in International Conference on Image Processing Theory, Tools and Applications (2017)

    Google Scholar 

  59. S. Balters, M. Steinert, Capturing emotion reactivity through physiology measurement as a foundation for affective engineering in engineering design science and engineering practices. J. Intell. Manuf. 28(7), 1585–1607 (2017)

    Article  Google Scholar 

  60. M. Kächele, M. Amirian, P. Thiam, P. Werner, S. Walter, G. Palm, F. Schwenker, Adaptive confidence learning for the personalization of pain intensity estimation systems. Evol. Syst. 8(1), 71–83 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

Peter Bellmann is supported by a scholarship of the Landesgraduiertenförderung Baden-Württemberg at Ulm University. Patrick Thiam is supported by the Federal Ministry of Education and Research (BMBF) within the project SenseEmotion. The work of Friedhelm Schwenker is partially supported by the Transregional Collaborative Research Centre SFB/TRR 62 Companion-Technology for Cognitive Technical Systems, funded by the German Research Foundation (DFG). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Tesla K40 GPU used for this research.

Author information

Authors and Affiliations

  1. Institute of Neural Information Processing, Ulm University, James-Franck-Ring, 89081, Ulm, Germany

    Peter Bellmann, Patrick Thiam & Friedhelm Schwenker

Authors
  1. Peter Bellmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patrick Thiam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Friedhelm Schwenker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Bellmann .

Editor information

Editors and Affiliations

  1. University of Alberta, Edmonton, Alberta, Canada

    Witold Pedrycz

  2. National Taiwan University of Science and Technology, Taipei, Taiwan

    Shyi-Ming Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Bellmann, P., Thiam, P., Schwenker, F. (2018). Multi-classifier-Systems: Architectures, Algorithms and Applications. In: Pedrycz, W., Chen, SM. (eds) Computational Intelligence for Pattern Recognition. Studies in Computational Intelligence, vol 777. Springer, Cham. https://doi.org/10.1007/978-3-319-89629-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-89629-8_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-89628-1

  • Online ISBN: 978-3-319-89629-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation