Large Scale Image Indexing Using Online Non-negative Semantic Embedding

  • Jorge A. Vanegas
  • Fabio A. González
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8258)


This paper presents a novel method to address the problem of indexing a large set of images taking advantage of associated multimodal content such as text or tags. The method finds relationships between the visual and text modalities enriching the image content representation to improve the performance of content-based image search.

This method finds a mapping that connects visual and text information that allows to project new (annotated and unannotated) images to the space defined by semantic annotations, this new representation can be used to search into the collection using a query-by-example strategy and to annotate new unannotated images. The principal advantage of the proposed method is its formulation as an online learning algorithm, which can scale to deal with large image collections. The experimental evaluation shows that the proposed method, in comparison with several baseline methods, is faster and consumes less memory, keeping a competitive performance in content-based image search.


Image Retrieval Nonnegative Matrix Factorization Mean Average Precision Semantic Space Stochastic Gradient Descent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Barnard, K., Duygulu, P., Forsyth, D., De Freitas, N., Blei, D.M., Kandola, J., Hofmann, T., Poggio, T., Shawe-Taylor, J.: Matching words and pictures. JMLR 3, 1107–1135 (2003)zbMATHGoogle Scholar
  2. 2.
    Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: COMPSTAT 2010, Paris, France, pp. 177–187. Springer (August 2010)Google Scholar
  3. 3.
    Bottou, L., LeCun, Y.: Large scale online learning. In: NIPS (2003)Google Scholar
  4. 4.
    Caicedo, J.C., BenAbdallah, J., González, F.A., Nasraoui, O.: Multimodal representation, indexing, automated annotation and retrieval of image collections via non-negative matrix factorization. Neurocomput. 76(1), 50–60 (2012)CrossRefGoogle Scholar
  5. 5.
    Caicedo, J.C., Cruz, A., Gonzalez, F.A.: Histopathology image classification using bag of features and kernel functions. In: Combi, C., Shahar, Y., Abu-Hanna, A. (eds.) AIME 2009. LNCS, vol. 5651, pp. 126–135. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  6. 6.
    Caicedo, J.C., González, F.A.: Online matrix factorization for multimodal image retrieval. In: Alvarez, L., Mejail, M., Gomez, L., Jacobo, J. (eds.) CIARP 2012. LNCS, vol. 7441, pp. 340–347. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  7. 7.
    Chandrika, P., Jawahar, C.V.: Multi modal semantic indexing for image retrieval. In: CIVR 2010, pp. 342–349. ACM, New York (2010)Google Scholar
  8. 8.
    Chen, Q., Tai, X., Jiang, B., Li, G., Zhao, J.: Medical image retrieval based on latent semantic indexing. In: CSSE 2008, pp. 561–564. IEEE Computer Society, Washington, DC (2008)Google Scholar
  9. 9.
    Cruz-Roa, A., Caicedo, J.C., González, F.A.: Visual pattern mining in histology image collections using bag of features. AIME 52(2), 91–106 (2011)Google Scholar
  10. 10.
    Fang, C., Torresani, L.: Measuring image distances via embedding in a semantic manifold. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part IV. LNCS, vol. 7575, pp. 402–415. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  11. 11.
    Guillaumin, M., Mensink, T., Verbeek, J., Schmid, C.: Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In: ICCV (2009)Google Scholar
  12. 12.
    Huiskes, M.J., Lew, M.S.: The mir flickr retrieval evaluation. In: MIR 2008. ACM, New York (2008)Google Scholar
  13. 13.
    Jen Lin, C.: Projected gradient methods for non-negative matrix factorization. Raport Instytutowy, Neural Computation (2007)Google Scholar
  14. 14.
    Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: NIPS, pp. 556–562. MIT Press (2000)Google Scholar
  15. 15.
    Makadia, A., Pavlovic, V., Kumar, S.: A new baseline for image annotation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 316–329. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  16. 16.
    Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. TPAMI 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  17. 17.
    Swain, M.J., Ballard, D.H.: Color indexing. IJCV 7, 11–32 (1991)CrossRefGoogle Scholar
  18. 18.
    Tsai, M.-H., Wang, J., Zhang, T., Gong, Y., Huang, T.S.: Learning semantic embedding at a large scale. In: ICIP, pp. 2497–2500 (2011)Google Scholar
  19. 19.
    Vanegas, J.A., Caicedo, J.C., González, F.A., Romero, E.: Histology image indexing using a non-negative semantic embedding. In: Müller, H., Greenspan, H., Syeda-Mahmood, T. (eds.) MCBR-CDS 2011. LNCS, vol. 7075, pp. 80–91. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  20. 20.
    Weston, J., Bengio, S., Usunier, N.: Large scale image annotation: Learning to rank with joint word-image embeddings. In: ECML (2010)Google Scholar
  21. 21.
    Yang, Z., Zhang, H., Yuan, Z., Oja, E.: Kullback-leibler divergence for nonnegative matrix factorization. In: Honkela, T. (ed.) ICANN 2011, Part I. LNCS, vol. 6791, pp. 250–257. Springer, Heidelberg (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Jorge A. Vanegas
    • 1
  • Fabio A. González
    • 1
  1. 1.MindLab Research GroupUniversidad Nacional de ColombiaBogotáColombia

Personalised recommendations