PATSI — Photo Annotation through Finding Similar Images with Multivariate Gaussian Models

  • Michal Stanek
  • Bartosz Broda
  • Halina Kwasnicka
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6375)


Automatic Image Annotation is important research topic in machine vision as it enables one to retrieve images from large databases by using textual queries. In recent years many machine learning techniques have been proposed to build detectors of concepts present on the images. In this paper we present a novel approach for image auto-annotation based on transfer of annotations from most similar images to the query image. We model image features by Multivariate Gaussian Distribution and measure distance between images by using Jensen-Shannon divergence. In spite of its simplicity, the proposed solution outperforms the state-of-the-art methods for image annotation and thus can be used as a baseline for developing other more elaborate methods.


Query Image Similar Image Image Annotation Semantic Label Automatic Image Annotation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Goodrum, A.: Image information retrieval: An overview of current research. Informing Science 3 (2000)Google Scholar
  2. 2.
    Carneiro, G., Chan, A., Moreno, P., Vasconcelos, N.: Supervised learning of semantic classes for image annotation and retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(3), 394–410 (2007)CrossRefGoogle Scholar
  3. 3.
    Hironobu, Y.M., Takahashi, H., Oka, R.: Image-to-word transformation based on dividing and vector quantizing images with words. In: Boltzmann machinesá, Neural Networks, vol. 4 (1999)Google Scholar
  4. 4.
    Duygulu, P., Barnard, K., de Freitas, J.F.G., Forsyth, D.A.: Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 97–112. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  5. 5.
    Lavrenko, V., Manmatha, R., Jeon, J.: A model for learning the semantics of picturesGoogle Scholar
  6. 6.
    Feng, S.L., Manmatha, R., Lavrenko, V.: Multiple bernoulli relevance models for image and video annotation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1002–1009 (2004)Google Scholar
  7. 7.
    Chang, E., Goh, K., Sychay, G., Wu, G.: Cbsa: content-based soft annotation for multimodal image retrieval using bayes point machines. IEEE Transactions on Circuits and Systems for Video Technology 13(1), 26–38 (2003)CrossRefGoogle Scholar
  8. 8.
    Cusano, C., Ciocca, G., Schettini, R.: Image annotation using svm. In: Proceedings of SPIE, vol. 5304, pp. 330–338 (2004)Google Scholar
  9. 9.
    Kwasnicka, H., Paradowski, M.: Resulted word counts optimization-a new approach for better automatic image annotation. Pattern Recogn. 41(12) (2008)Google Scholar
  10. 10.
    Carneiro, G., Vasconcelos, N.: A database centric view of semantic image annotation and retrieval. In: SIGIR 2005: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 559–566. ACM, New York (2005)CrossRefGoogle Scholar
  11. 11.
    Kwasnicka, H., Paradowski, M.: Multiple class machine learning approach for an image auto-annotation problem. In: ISDA 2006: Proceedings of the Sixth International Conference on Intelligent Systems Design and Applications, Washington, DC, USA, pp. 347–352. IEEE Computer Society, Los Alamitos (2006)CrossRefGoogle Scholar
  12. 12.
    Jin, Y., Khan, L., Wang, L., Awad, M.: Image annotations by combining multiple evidence & wordnet. In: MULTIMEDIA 2005: Proceedings of the 13th annual ACM international conference on Multimedia. ACM, New York (2005)Google Scholar
  13. 13.
    Llorente, A., Motta, E., Rüger, S.: Image annotation refinement using web-based keyword correlation. In: Chua, T.-S., Kompatsiaris, Y., Mérialdo, B., Haas, W., Thallinger, G., Bailer, W. (eds.) SAMT 2009. LNCS, vol. 5887, pp. 188–191. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  14. 14.
    Makadia, A., Pavlovic, V., Kumar, S.: A new baseline for image annotation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 316–329. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  15. 15.
    McLachlan, G.J., Krishnan, T.: The EM Algorithm and Extensions, 2nd edn. Wiley Series in Probability and Statistics. Wiley-Interscience, Hoboken (March 2008)Google Scholar
  16. 16.
    Paradowski, M.: Metody automatycznej anotacji jako wydajne narzedzie opisuj”ace kolekcje obrazow. PhD thesis, Wrocław University of Technology (2008)Google Scholar
  17. 17.
    Icpr 2004 image database (2004),
  18. 18.
    Grubinger, M., Clough, P.D., Henning, M., Thomas, D.: The iapr benchmark: A new evaluation resource for visual information systems. In: International Conference on Language Resources and Evaluation, Genoa, Italy (May 2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Michal Stanek
    • 1
  • Bartosz Broda
    • 1
  • Halina Kwasnicka
    • 1
  1. 1.Institute of InformaticsWrocław University of Technology 

Personalised recommendations