Predicting Emotional States of Images Using Bayesian Multiple Kernel Learning

  • He Zhang
  • Mehmet Gönen
  • Zhirong Yang
  • Erkki Oja
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8228)


Images usually convey information that can influence people’s emotional states. Such affective information can be used by search engines and social networks for better understanding the user’s preferences. We propose here a novel Bayesian multiple kernel learning method for predicting the emotions evoked by images. The proposed method can make use of different image features simultaneously to obtain a better prediction performance, with the advantage of automatically selecting important features. Specifically, our method has been implemented within a multilabel setup in order to capture the correlations between emotions. Due to its probabilistic nature, our method is also able to produce probabilistic outputs for measuring a distribution of emotional intensities. The experimental results on the International Affective Picture System (IAPS) dataset show that the proposed approach achieves a bette classification performance and provides a more interpretable feature selection capability than the state-of-the-art methods.


Image emotion low-level image features multiview learning multiple kernel learning variational approximation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Albert, J.H., Chib, S.: Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association 88(422), 669–679 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Beal, M.J.: Variational Algorithms for Approximate Bayesian Inference. Ph.D. thesis, The Gatsby Computational Neuroscience Unit, University College London (2003)Google Scholar
  3. 3.
    Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273–297 (1995)zbMATHGoogle Scholar
  4. 4.
    Gönen, M., Alpaydın, E.: Multiple kernel learning algorithms. Journal of Machine Learning Research 12, 2211–2268 (2011)Google Scholar
  5. 5.
    Hanjalic, A.: Extracting moods from pictures and sounds: Towards truly personalized TV. IEEE Signal Processing Magazine 23(2), 90–100 (2006)CrossRefGoogle Scholar
  6. 6.
    Laaksonen, J., Koskela, M., Oja, E.: PicSOM-self-organizing image retrieval with MPEG-7 content descriptors. IEEE Transactions on Neural Networks 13(4), 841–853 (2002)CrossRefGoogle Scholar
  7. 7.
    Lawrence, N.D., Jordan, M.I.: Semi-supervised learning via Gaussian processes. In: Advances in Neural Information Processing Systems 17, pp. 753–760 (2005)Google Scholar
  8. 8.
    Lu, X., Suryanarayan, P., Adams Jr., R.B., Li, J., Newman, M.G., Wang, J.Z.: On shape and the computability of emotions. In: Proceedings of the International Conference on Multimedia, pp. 229–238 (2012)Google Scholar
  9. 9.
    Machajdik, J., Hanbury, A.: Affective image classification using features inspired by psychology and art theory. In: Proceedings of the International Conference on Multimedia, pp. 83–92 (2010)Google Scholar
  10. 10.
    Mikels, J., Fredrickson, B., Larkin, G., Lindberg, C., Maglio, S., Reuter-Lorenz, P.: Emotional category data on images from the International Affective Picture System. Behavior Research Methods 37(4), 626–630 (2005)CrossRefGoogle Scholar
  11. 11.
    Picard, R.: Affective Computing. MIT Press (1997)Google Scholar
  12. 12.
    Sjöberg, M., Muurinen, H., Laaksonen, J., Koskela, M.: PicSOM experiments in TRECVID 2006. In: Proceedings of the TRECVID 2006 Workshop (2006)Google Scholar
  13. 13.
    Zhang, H., Augilius, E., Honkela, T., Laaksonen, J., Gamper, H., Alene, H.: Analyzing emotional semantics of abstract art using low-level image features. In: Gama, J., Bradley, E., Hollmén, J. (eds.) IDA 2011. LNCS, vol. 7014, pp. 413–423. Springer, Heidelberg (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • He Zhang
    • 1
  • Mehmet Gönen
    • 1
  • Zhirong Yang
    • 1
  • Erkki Oja
    • 1
  1. 1.Department of Information and Computer ScienceAalto University School of ScienceEspooFinland

Personalised recommendations