Advertisement

Multimedia Tools and Applications

, Volume 56, Issue 1, pp 131–153 | Cite as

Inferring photographic location using geotagged web images

  • Dhiraj Joshi
  • Andrew Gallagher
  • Jie Yu
  • Jiebo Luo
Article

Abstract

Geotagging has become a recent phenomenon that allows users to visualize and manage photo collections in many new and interesting ways. Unfortunately, manual geotagging of a large collection of pictures on the globe is still a time-consuming and laborious task even though geotagging devices are gradually being adopted. At the same time, there exist billions of legacy pictures taken before the onset of geotagging. In recent times, large collections of Web images have been found to facilitate a number of image understanding tasks including geolocation estimation. In this paper, we leverage user tags along with image content to infer the geolocation of images. Our model builds upon the fact that the visual content and user tags of pictures can together provide significant hints about their geolocations. Using a collection of over a million geotagged pictures, we build location probability maps for commonly used image tags over the entire globe. These maps reflect the collective picture-taking and tagging behaviors of thousands of users from all over the world. We further study the geographic entropy and frequency of user tags as geo-inference features and investigate the usefulness of using these features for selecting geographically meaningful annotations. On the other hand, visual content matching is performed using multiple feature descriptors including tiny images, color histograms, GIST features, and bags of textons. Finally, visual KNN matching based geographic mapping scheme is integrated with tag location probability maps to form a strong geo-inference engine. Experiments have shown improvements over geolocation inference performed using either modality alone.

Keywords

Geotagged images Location probability maps KNN Visual matching 

References

  1. 1.
    Ames M, Naaman M (2007) Why we tag: motivations for annotation in mobile and online media. In Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsGoogle Scholar
  2. 2.
    Amitay E, Har’El N, Sivan R, Soffer A (2004) Web-a-where: geotagging web content. In Proceedings of the ACM SIGIR Conference on Research and Development in Information RetrievalGoogle Scholar
  3. 3.
    Chen Y, Chen XY, Rao FY, Yu XL, Li Y, Liu D (2004) LORE: an infrastructure to support location-aware services. IBM J Res Develop 48(5/6):601–616CrossRefGoogle Scholar
  4. 4.
    Cilibrasi RL, Vitanyi PMB (2007) The Google similarity distance. IEEE Trans Knowl Data Eng 19(3):370–383CrossRefGoogle Scholar
  5. 5.
    Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619CrossRefGoogle Scholar
  6. 6.
    Crandall D, Backstrom L, Huttenlocher D, Kleinberg J (2009) Mapping the world’s photos, WWWGoogle Scholar
  7. 7.
    Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(65)Google Scholar
  8. 8.
    Dubinko M, Kumar R, Magnani J, Novak J, Raghavan P, Tomkins A (2006) Visualizing tags over time. In Proceedings of the World Wide WebGoogle Scholar
  9. 9.
    Gallagher A, Joshi D,Yu J, Luo J (2009) Geolocation inference from image content and user tags, CVPR Workshop in Internet VisionGoogle Scholar
  10. 10.
    Hays J, Efros A (2008) IM2GPS: estimating geographic information from a single image. IEEE International Conference on Computer Vision and Pattern RecognitionGoogle Scholar
  11. 11.
    Hinze A, Voisard A (2003) Location and time-based information delivery in tourism. Advances in Spatial and Temporal Databases. Lect NotesComput Sci 2750:489–507CrossRefGoogle Scholar
  12. 12.
    Jacobs N, Satkin S, Roman N, Speyer R, Pless R (2007) Geolocating static cameras. In Proceedings of the International Conference on Computer VisionGoogle Scholar
  13. 13.
    Jaffe A, Tassa T, Davis M (2006) Generating summaries and visualization for large collections of geo-referenced photographs. ACM International Workshop on Multimedia Information RetrievalGoogle Scholar
  14. 14.
    Jain V, Singhal A (2008) Selective hidden random fields: exploiting domain specific saliency for event classification. In Proceedings of the International Conference on Computer Vision and Pattern RecognitionGoogle Scholar
  15. 15.
    Joshi D, Luo J (2008) Inferring generic activities and events from image content and bags of geo-tags. ACM International Conference on Image and Video RetrievalGoogle Scholar
  16. 16.
    Kennedy L, Naaman M, Ahern S, Nair R, Rattenbury T (2007) How Flickr helps us make sense of the world: context and content in community-contributed media collections. ACM International Conference on MultimediaGoogle Scholar
  17. 17.
    Kosecka J, Zhang W (2002) Video compass. In Proceedings of European Conference on Computer VisionGoogle Scholar
  18. 18.
    Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories, In Proceedings of International Conference on Computer Vision and Pattern RecognitionGoogle Scholar
  19. 19.
    Li L-J, Fei-Fei L (2007) What, where and who? Classifying event by scene and object recognition. In Proceedings of International Conference on Computer VisionGoogle Scholar
  20. 20.
    Liu L, Wolfson O, Yin H (2006) Extracting semantic location from outdoor positioning systems. In Proceedings of the IEEE International Conference on Mobile Data Management, 2006.Google Scholar
  21. 21.
    Li J, Chang S-F, Lesk M, Leinhart R, Luo J, Smeulders AW (2007) New challenges in multimedia research for the increasingly connected and fast growing digital society. In Proceedings of the ACM International Workshop on Multimedia Information RetrievalGoogle Scholar
  22. 22.
    Luo J, Boutell M, Brown C (2006) Pictures are not taken in a vacuum: an overview of exploiting context for semantic scene content understanding. IEEE Signal Process Mag 23(2):101–114CrossRefGoogle Scholar
  23. 23.
    Luo J, Joshi D, Yu J, Hao W (2008) Event recognition: viewing the world with a third eye. ACM International Conference on MultimediaGoogle Scholar
  24. 24.
    Pedersen T, Patwardhan S, Michelizzi J (2004) WordNet similarity—measuring the relatedness of concepts. In Proceedings of the Nineteenth National Conference on Artificial IntelligenceGoogle Scholar
  25. 25.
    Quack T, Leibe B, Van Gool L (2008) World-scale mining of objects and events from community photo collections, In Proceedings of the ACM Conference on Image and Video RetrievalGoogle Scholar
  26. 26.
    Schiller JH, Voisard A (2004) Location-based services, Morgan KaufmannGoogle Scholar
  27. 27.
    Schindler G, Brown M, Szeliski R (2007) City-scale location recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern RecognitionGoogle Scholar
  28. 28.
    Schindler G, Krishnamurthy P, Lublinerman R, Liu Y, Dellaert F (2008) Detecting and matching repeated patterns for automatic geo-tagging in urban environments. IEEE International Conference on Computer Vision and Pattern RecognitionGoogle Scholar
  29. 29.
    Simon I, Snavely N, Seitz S (2007) Scene summarization for online image collections. In Proceedings of IEEE International Conference on Computer VisionGoogle Scholar
  30. 30.
    Torralba A, Fergus R, Freeman WT (2007) Tiny images. Technical Report MIT-CSAIL-TR-2007-024Google Scholar
  31. 31.
    Wolf L, Bileschi S (2006) A critical view of context. Int J Comput Vis 68(1):43–52CrossRefGoogle Scholar
  32. 32.
    Yu J, Luo J (2008) Leveraging probabilistic season and location context models for scene understanding. In Proceedings of the ACM International Conference on Image and Video RetrievalGoogle Scholar
  33. 33.
    Zheng Y-T, Zhao M, Song Y, Adam H, Buddemeier U, Bissacco A, Brucher F, Chua T-S, Neven H (2009) Tour the world: building a webscale landmark recognition engine. In Proceedings of IEEE Conference on Computer Vision and Pattern RecognitionGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Dhiraj Joshi
    • 1
  • Andrew Gallagher
    • 1
  • Jie Yu
    • 1
  • Jiebo Luo
    • 1
  1. 1.Kodak Research LaboratoriesEastman Kodak CompanyRochesterUSA

Personalised recommendations