Evaluating Temporal Information for Social Image Annotation and Retrieval

  • Tiberio Uricchio
  • Lamberto Ballan
  • Marco Bertini
  • Alberto Del Bimbo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8156)


Can we use the temporal evolution of annotations in Web images to improve tasks such as annotation, indexing and retrieval? This important question is the main motivation for this work. Typically visual content, text and metadata, are used to improve these tasks. A characteristic that has received less attention, so far, is the temporal aspect of social media production and tagging. The main contribution of this paper is a thorough analysis of the temporal aspects of two popular datasets commonly used for tasks such as tag ranking, tag suggestion and tag refinement, namely NUS-WIDE and MIR-Flickr-1M. The correlation of the time series of the tags with Google searches shows that for certain concepts web information sources may be beneficial to annotate social media.


Temporal information image annotation image retrieval image tagging social media 


  1. 1.
    von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: ACM CHI (2004)Google Scholar
  2. 2.
    Alonso, O., Gertz, M., Baeza-Yates, R.: On the value of temporal information in information retrieval. SIGIR Forum 41(2), 35–41 (2007)CrossRefGoogle Scholar
  3. 3.
    Uricchio, T., Ballan, L., Bertini, M., Del Bimbo, A.: An evaluation of nearest-neighbor methods for tag refinement. In: IEEE ICME (2013)Google Scholar
  4. 4.
    Choi, H., Varian, H.: Predicting the present with Google Trends. Tech. rep., Google (2011)Google Scholar
  5. 5.
    Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: NUS-WIDE: A real-world web image database from National University of Singapore. In: ACM CIVR (2009)Google Scholar
  6. 6.
    Cohen, J.: Statistical power analysis for the behavioral sciences. Routledge Academic (1988)Google Scholar
  7. 7.
    Ginsberg, J., Mohebbi, M.H., Patel, R.S., Brammer, L., Smolinski, M.S., Brilliant, L.: Detecting influenza epidemics using search engine query data. Nature 457(7232), 1012–1014 (2009)CrossRefGoogle Scholar
  8. 8.
    Huiskes, M.J., Lew, M.S.: The MIR Flickr retrieval evaluation. In: ACM MIR (2008)Google Scholar
  9. 9.
    Huiskes, M.J., Thomee, B., Lew, M.S.: New trends and ideas in visual concept detection: the MIR Flickr retrieval evaluation initiative. In: ACM MIR, pp. 527–536 (2010)Google Scholar
  10. 10.
    Jin, X., Gallagher, A., Cao, L., Luo, J., Han, J.: The wisdom of social multimedia: using Flickr for prediction and forecast. In: ACM MM, pp. 1235–1244 (2010)Google Scholar
  11. 11.
    Kennedy, L.S., Chang, S.F., Kozintsev, I.V.: To search or to label? Predicting the performance of search-based automatic image classifiers. In: ACM MIR (2006)Google Scholar
  12. 12.
    Kim, G., Fei-Fei, L., Xing, E.P.: Web image prediction using multivariate point processes. In: ACM SIGKDD, pp. 1068–1076 (2012)Google Scholar
  13. 13.
    Kim, G., Xing, E.P.: Time-sensitive web image ranking and retrieval via dynamic multi-task regression. In: ACM WSDM, pp. 163–172 (2013)Google Scholar
  14. 14.
    Kim, G., Xing, E.P., Torralba, A.: Modeling and analysis of dynamic behaviors of web image collections. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 85–98. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  15. 15.
    Li, X., Snoek, C.G.M., Worring, M.: Learning social tag relevance by neighbor voting. IEEE Transactions on Multimedia 11(7), 1310–1322 (2009)CrossRefGoogle Scholar
  16. 16.
    Liu, D., Hua, X.S., Yang, L., Wang, M., Zhang, H.J.: Tag ranking. In: WWW (2009)Google Scholar
  17. 17.
    Liu, D., Yan, S., Hua, X.S., Zhang, H.J.: Image retagging using collaborative tag propagation. IEEE Transactions on Multimedia 13(4), 702–712 (2011)CrossRefGoogle Scholar
  18. 18.
    Rattenbury, T., Good, N., Naaman, M.: Towards automatic extraction of event and place semantics from flickr tags. In: ACM SIGIR, pp. 103–110 (2007)Google Scholar
  19. 19.
    Sigurbjörnsson, B., van Zwol, R.: Flickr tag recommendation based on collective knowledge. In: WWW, pp. 327–336 (2008)Google Scholar
  20. 20.
    Sizov, S.: Geofolk: latent spatial semantics in web 2.0 social media. In: ACM WSDM, pp. 281–290 (2010)Google Scholar
  21. 21.
    Sundaram, H., Xie, L., De Choudhury, M., Lin, Y.R., Natsev, A.: Multimedia semantics: Interactions between content and community. Proceedings of the IEEE 100(9), 2737–2758 (2012)CrossRefGoogle Scholar
  22. 22.
    Team, R.C.: R: A language and environment for statistical computing. vienna, austria: R foundation for statistical computing; 2008 (2011)Google Scholar
  23. 23.
    Zhu, G., Yan, S., Ma, Y.: Image tag refinement towards low-rank, content-tag prior and error sparsity. In: ACM Multimedia (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Tiberio Uricchio
    • 1
  • Lamberto Ballan
    • 1
  • Marco Bertini
    • 1
  • Alberto Del Bimbo
    • 1
  1. 1.Media Integration and Communication Center (MICC)Università degli Studi di FirenzeItaly

Personalised recommendations