Classifier Fusion Applied to Facial Expression Recognition: An Experimental Comparison

  • Martin Schels
  • Christian Thiel
  • Friedhelm Schwenker
  • Günther Palm
Part of the Cognitive Systems Monographs book series (COSMOS, volume 6)


In this paper classifier fusion approaches are investigated through numerical evaluation. For this purpose a multi classifier architecture for the recognition of human facial expressions in image sequences has been constructed on characteristic facial regions and three different feature types (principal components, orientation histograms of static images and temporal features based on optical flow). Classifier fusion is applied to the individual channels established by feature principle and facial region, which are addressed to by individual classifiers. The available combinations of classifier outputs are examined and it is investigated how combining classifiers can lead to more appropriate results. The stability of fusion regarding varying classifier combinations is studied and the fused classifier output is compared to the human view on the data.


Facial Expression Recognition Rate Confusion Matrix Gesture Recognition Fusion Rule 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bayerl, P., Neumann, H.: Disambiguating visual motion through contextual feedback modulation. Neural Comput. 16, 2041–2066 (2004)zbMATHCrossRefGoogle Scholar
  2. 2.
    Bayerl, P., Neumann, H.: A fast biologically inspired algorithm for recurrent motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 246–260 (2007)CrossRefGoogle Scholar
  3. 3.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)zbMATHCrossRefGoogle Scholar
  4. 4.
    Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., Taylor, J.: Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine 18, 32–80 (2001)CrossRefGoogle Scholar
  5. 5.
    Freeman, W.T., Roth, M.: Orientation Histograms for Hand Gesture Recognition. In: International Workshop on Automatic Face and Gesture Recognition, pp. 296–301 (1994)Google Scholar
  6. 6.
    Kanade, T., Cohn, J., Tian, Y.L.: Comprehensive database for facial expression analysis. In: Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2000), pp. 46–53 (2000)Google Scholar
  7. 7.
    Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20, 226–239 (1998)CrossRefGoogle Scholar
  8. 8.
    Kuncheva, L., Bezdek, J.C., Duin, R.P.W.: Decision templates for multiple classifier fusion: an experimental comparison. Pattern Recognition 34, 299–314 (2001)zbMATHCrossRefGoogle Scholar
  9. 9.
    Lang, P.J.: The emotion probe. studies of motivation and attention. The American Psychologist 50, 372–385 (1995)Google Scholar
  10. 10.
    Poggio, T., Girosi, F.: A theory of networks for approximation and learning. Laboratory, Massachusetts Institute of Technology 1140 (1989)Google Scholar
  11. 11.
    Riedmiller, M., Braun, H.: RPROP – description and implementation details. Technical report, Universität Karlsruhe (1994)Google Scholar
  12. 12.
    Roli, F., Kittler, J., Fumera, G., Muntoni, D.: An experimental comparison of classifier fusion rules for multimodal personal identity verification systems. In: Roli, F., Kittler, J. (eds.) MCS 2002. LNCS, vol. 2364, pp. 325–336. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  13. 13.
    Rosenblum, M., Yacoob, Y., Davis, L.: Human expression recognition from motion using a radial basis function network architecture. IEEE Transactions on Neural Networks 7, 1121–1138 (1996)CrossRefGoogle Scholar
  14. 14.
    Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (Adaptive Computation and Machine Learning). The MIT Press, Cambridge (2001)Google Scholar
  15. 15.
    Schwenker, F., Dietrich, C., Thiel, C., Palm, G.: Learning of decision fusion mappings for pattern recognition. International Journal on Artificial Intelligence and Machine Learning (AIML) 6, 17–21 (2006)Google Scholar
  16. 16.
    Schwenker, F., Sachs, A., Palm, G., Kestler, H.A.: Orientation Histograms for Face Recognition. In: Schwenker, F., Marinai, S. (eds.) ANNPR 2006. LNCS (LNAI), vol. 4087, pp. 253–259. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    Thiel, C., Giacco, F., Schwenker, F., Palm, G.: Comparison of Neural Classification Algorithms Applied to Land Cover Mapping. In: Proceedings of the 18th Italian Workshop on Neural Networks, WIRN 2008. IOS Press, Amsterdam (2008)Google Scholar
  18. 18.
    Turk, M., Pentland, A.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 3, 71–86 (1991)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Martin Schels
    • 1
  • Christian Thiel
    • 1
  • Friedhelm Schwenker
    • 1
  • Günther Palm
    • 1
  1. 1.Institute of Neural Information ProcessingUniversity of UlmUlmGermany

Personalised recommendations