On Instance Selection in Audio Based Emotion Recognition

  • Sascha Meudt
  • Friedhelm Schwenker
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7477)


Affective computing aim to provide simpler and more natural interfaces for human-computer interaction applications, e.g. recognizing automatically the emotional status of the user based on facial expressions or speech is important to model user as complete as possible in order to develop human-computer interfaces that are able to respond to the user’s action or behavior in an appropriate manner. In this paper we focus on audio-based emotion recognition. Data sets employed for the statistical evaluation have been collected through Wizard-of-Oz experiments. The emotional labels have been are defined through the experimental set up therefore given on a relatively coarse temporal scale (a few minutes) which This global labeling concept might lead to miss-labeled data at smaller time scales, for instance for window sizes uses in audio analysis (less than a second). Manual labeling at these time scales is very difficult not to say impossible, and therefore our approach is to use the globally defined labels in combination with instance/sample selection methods. In such an instance selection approach the task is to select the most relevant and discriminative data of the training set by using a pre-trained classifier. Mel-Frequency Cepstral Coefficients (MFCC) features are used to extract relevant features, and probabilistic support vector machines (SVM) are applied as base classifiers in our numerical evaluation. Confidence values to the samples of the training set are assigned through the outputs of the probabilistic SVM.


Emotion Recognition Human Computer Interaction Instance Selection Active Learning 


  1. 1.
    Bishop, C.: Pattern recognition and machine learning, vol. 4. Springer, New York (2006)zbMATHGoogle Scholar
  2. 2.
    Brighton, H., Mellish, C.: Advances in instance selection for instance-based learning algorithms. Data Mining and Knowledge Discovery 6(2), 153–172 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Domingo, C., Gavaldà, R., Watanabe, O.: Adaptive sampling methods for scaling up knowledge discovery algorithms. Data Mining and Knowledge Discovery 6(2), 131–152 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Esparza, J., Scherer, S., Brechmann, A., Schwenker, F.: Automatic emotion classification vs. human perception: Comparing machine performance to the human benchmark. In: International Conference on Information Science, Signal Processing and Their Applications (ISSPA 2012), pp. 1286–1291 (2012)Google Scholar
  5. 5.
    Esparza, J., Scherer, S., Schwenker, F.: Studying Self- and Active-Training Methods for Multi-feature Set Emotion Recognition. In: Schwenker, F., Trentin, E. (eds.) PSL 2011. LNCS, vol. 7081, pp. 19–31. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  6. 6.
    Glodek, M., Tschechne, S., Layher, G., Schels, M., Brosch, T., Scherer, S., Kchele, M., Schmidt, M., Neumann, H., Palm, G., Schwenker, F.: Multiple classifier systems for the classification of audio-visual emotional states. In: 1st International Audio/Visual Emotion Challenge and Workshop (2011)Google Scholar
  7. 7.
    de Haro-García, A., García-Pedrajas, N., del Castillo, J.A.R.: Large scale instance selection by means of federal instance selection. Data Mining and Knowledge Engineering 75, 58–77 (2012)CrossRefGoogle Scholar
  8. 8.
    Kelley, J.: An iterative design methodology for user-friendly natural language office information applications. ACM Transactions on Information Systems (TOIS) 2(1), 26–41 (1984)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Liu, H., Motoda, H.: Instance Selection and Construction for Data Mining. Kluwer Academic Publishers, Norwell (2001)Google Scholar
  10. 10.
    Liu, H., Motoda, H.: On issues of instance selection. Data Mining and Knowledge Discovery, 115–130 (2002)Google Scholar
  11. 11.
    Madigan, D., Raghavan, N., DuMouchel, W., Nason, M., Posse, C., Ridgeway, G.: Likelihood-based data squashing: A modeling approach to instance construction. Data Mining and Knowledge Discovery 6(2), 173–190 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Meudt, S., Bigalke, L., Schwenker, F.: ATLAS – an annotation tool for HCI data utilizing machine learning methods. In: Proceedings of the 4th Internantional Conference on Applied Human Factors and Ergonomics, AHFE 2012 (in print, 2012)Google Scholar
  13. 13.
    Olvera-Lpez, J.A., Carrasco-Ochoa, J.A., Trinidad, J.F.M., Kittler, J.: A review of instance selection methods. Artificial Intelligence Reviews, 133–143 (2010)Google Scholar
  14. 14.
    Platt, J.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers 10(3), 61–74 (1999)Google Scholar
  15. 15.
    Reinartz, T.: A unifying view on instance selection. Data Mining and Knowledge Discovery 6(2), 191–210 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Russell, J.A., Mehrabian, A.: Evidence for a three-factor theory of emotions. Journal of Research in Personality 11(3), 273–294 (1977)CrossRefGoogle Scholar
  17. 17.
    Schels, M., Kächele, M., Hrabal, D., Walter, S., Traue, H.C., Schwenker, F.: Classification of Emotional States in a Woz Scenario Exploiting Labeled and Unlabeled Bio-physiological Data. In: Schwenker, F., Trentin, E. (eds.) PSL 2011. LNCS, vol. 7081, pp. 138–147. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  18. 18.
    Scherer, S., Glodek, M., Schwenker, F., Campbell, N., Palm, G.: Spotting laughter in natural multiparty conversations: A comparison of automatic online and offline approaches using audiovisual data. TiiS 2(1), 4 (2012)CrossRefGoogle Scholar
  19. 19.
    Walter, S., Scherer, S., Schels, M., Glodek, M., Hrabal, D., Schmidt, M., Böck, R., Limbrecht, K., Traue, H.C., Schwenker, F.: Multimodal Emotion Classification in Naturalistic User Behavior. In: Jacko, J.A. (ed.) HCII 2011, Part III. LNCS, vol. 6763, pp. 603–611. Springer, Heidelberg (2011), CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Sascha Meudt
    • Friedhelm Schwenker

      There are no affiliations available

      Personalised recommendations