Advertisement

Affective Abstract Image Classification and Retrieval Using Multiple Kernel Learning

  • He Zhang
  • Zhirong Yang
  • Mehmet Gönen
  • Markus Koskela
  • Jorma Laaksonen
  • Timo Honkela
  • Erkki Oja
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8228)

Abstract

Emotional semantic image retrieval systems aim at incorporating the user’s affective states for responding adequately to the user’s interests. One challenge is to select features specific to image affect detection. Another challenge is to build effective learning models or classifiers to bridge the so-called “affective gap”. In this work, we study the affective classification and retrieval of abstract images by applying multiple kernel learning framework. An image can be represented by different feature spaces and multiple kernel learning can utilize all these feature representations simultaneously (i.e., multiview learning), such that it jointly learns the feature representation weights and corresponding classifier in an intelligent manner. Our experimental results on two abstract image datasets demonstrate the advantage of the multiple kernel learning framework for image affect detection in terms of feature selection, classification performance, and interpretation.

Keywords

Image affect multiple kernel learning group lasso low-level image features image classification and retrieval 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bach, F.: Consistency of the group lasso and multiple kernel learning. Journal of Machine Learning Research 9, 1179–1225 (2008)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Bach, F., Lanckriet, G., Jordan, M.: Multiple kernel learning, conic duality, and the SMO algorithm. In: Proceedings of the 21st International Conference on Machine Learning (ICML). ACM (2004)Google Scholar
  3. 3.
    Chang, C., Lin, C.: Libsvm: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST) 2(3) (2011)Google Scholar
  4. 4.
    Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273–297 (1995)zbMATHGoogle Scholar
  5. 5.
    Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys 40(2), 5 (2008)CrossRefGoogle Scholar
  7. 7.
    Gönen, M., Alpaydın, E.: Multiple kernel learning algorithms. Journal of Machine Learning Research 12, 2211–2268 (2011)Google Scholar
  8. 8.
    Hanjalic, A.: Extracting moods from pictures and sounds: Towards truly personalized TV. IEEE Signal Processing Magazine 23(2), 90–100 (2006)CrossRefGoogle Scholar
  9. 9.
    Honkela, T., Lindh-Knuutila, T., Lagus, K.: Measuring adjective spaces. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds.) ICANN 2010, Part I. LNCS, vol. 6352, pp. 351–355. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    Laaksonen, J., Koskela, M., Oja, E.: PicSOM-self-organizing image retrieval with MPEG-7 content descriptors. IEEE Transactions on Neural Networks 13(4), 841–853 (2002)CrossRefGoogle Scholar
  11. 11.
    Machajdik, J., Hanbury, A.: Affective image classification using features inspired by psychology and art theory. In: Proceedings of the International Conference on Multimedia, pp. 83–92. ACM (2010)Google Scholar
  12. 12.
    Ou, L., Luo, M.R., Woodcock, A., Wright, A.: A study of colour emotion and colour preference. Part I: Colour emotions for single colours. Color Research & Application 29(3), 232–240 (2004)CrossRefGoogle Scholar
  13. 13.
    Palatucci, M., Pomerleau, D., Hinton, G.E., Mitchell, T.M.: Zero-shot learning with semantic output codes. In: Advances in Neural Information Processing Systems (NIPS), pp. 1410–1418 (2009)Google Scholar
  14. 14.
    Shamir, L., Macura, T., Orlov, N., Eckley, D.M., Goldberg, I.G.: Impressionism, expressionism, surrealism: Automated recognition of painters and schools of art. ACM Transactions on Applied Perception 7(2) (2010)Google Scholar
  15. 15.
    Sjöberg, M., Muurinen, H., Laaksonen, J., Koskela, M.: PicSOM experiments in TRECVID 2006. In: Proceedings of the TRECVID 2006 Workshop (2006)Google Scholar
  16. 16.
    Vedaldi, A., Gulshan, V., Varma, M., Zisserman, A.: Multiple kernels for object detection. In: Proceedings of the 12th International Conference on Computer Vision (ICCV), pp. 606–613. IEEE (2009)Google Scholar
  17. 17.
    Wang, W., He, Q.: A survey on emotional semantic image retrieval. In: Proceedings of 15th IEEE International Conference on Image Processing, pp. 117–120 (2008)Google Scholar
  18. 18.
    Xu, Z., Jin, R., Yang, H., King, I., Lyu, M.: Simple and efficient multiple kernel learning by group lasso. In: Proceedings of the 27th International Conference on Machine Learning (ICML), pp. 1175–1182 (2010)Google Scholar
  19. 19.
    Zhang, H., Augilius, E., Honkela, T., Laaksonen, J., Gamper, H., Alene, H.: Analyzing emotional semantics of abstract art using low-level image features. In: Gama, J., Bradley, E., Hollmén, J. (eds.) IDA 2011. LNCS, vol. 7014, pp. 413–423. Springer, Heidelberg (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • He Zhang
    • 1
  • Zhirong Yang
    • 1
  • Mehmet Gönen
    • 1
  • Markus Koskela
    • 1
  • Jorma Laaksonen
    • 1
  • Timo Honkela
    • 1
  • Erkki Oja
    • 1
  1. 1.Department of Information and Computer ScienceAalto University School of ScienceEspooFinland

Personalised recommendations