Advertisement

GeoInformatica

, Volume 20, Issue 4, pp 829–857 | Cite as

Efficient indexing and retrieval of large-scale geo-tagged video databases

  • Ying Lu
  • Cyrus Shahabi
  • Seon Ho Kim
Article

Abstract

We are witnessing a significant growth in the number of smartphone users and advances in phone hardware and sensor technology. In conjunction with the popularity of video applications such as YouTube, an unprecedented number of user-generated videos (UGVs) are being generated and consumed by the public, which leads to a Big Data challenge in social media. In a very large video repository, it is difficult to index and search videos in their unstructured form. However, due to recent development, videos can be geo-tagged (e.g., locations from GPS receiver and viewing directions from digital compass) at the acquisition time, which can provide potential for efficient management of video data. Ideally, each video frame can be tagged by the spatial extent of its coverage area, termed Field-Of-View (FOV). This effectively converts a challenging video management problem into a spatial database problem. This paper attacks the challenges of large-scale video data management using spatial indexing and querying of FOVs, especially maximally harnessing the geographical properties of FOVs. Since FOVs are shaped similar to slices of pie and contain both location and orientation information, conventional spatial indexes, such as R-tree, cannot index them efficiently. The distribution of UGVs’ locations is non-uniform (e.g., more FOVs in popular locations). Consequently, even multilevel grid-based indexes, which can handle both location and orientation, have limitations in managing the skewed distribution. Additionally, since UGVs are usually captured in a casual way with diverse setups and movements, no a priori assumption can be made to condense them in an index structure. To overcome the challenges, we propose a class of new R-tree-based index structures that effectively harness FOVs’ camera locations, orientations and view-distances, in tandem, for both filtering and optimization. We also present novel search strategies and algorithms for efficient range and directional queries on our indexes. Our experiments using both real-world and large synthetic video datasets (over 30 years’ worth of videos) demonstrate the scalability and efficiency of our proposed indexes and search algorithms.

Keywords

Geo-tag Index Video retrieval Scalability User-generated video 

References

  1. 1.
    Ay AS, Zimmermann R, Kim SH (2008) Viewable scene modeling for geospatial video search. In: ACM international conference on MM, pp 309–318Google Scholar
  2. 2.
    Ay SA, Kim SH, Zimmermann R (2010) Generating synthetic meta-data for georeferenced video management. In: the SIGSPATIAL, pp 280–289Google Scholar
  3. 3.
    Ay SA, Zimmermann R, Kim SH (2010) Relevance ranking in georeferenced video search. ACM Multimed Syst (MMSys) 16(2):105–125CrossRefGoogle Scholar
  4. 4.
    Gilboa-Solomon F, Ashour G, Azulai O (2013) Efficient storage and retrieval of geo-referenced video from moving sensors. In: Proceedings of the 21st ACM SIGSPATIAL international conference on advances in geographic information systems, SIGSPATIAL’13. ACM, New York, pp 404–407Google Scholar
  5. 5.
    Guttman A (1984) R-trees: a dynamic index structure for spatial searching. In: SIGMOD, pp 47–57Google Scholar
  6. 6.
    Jain V, Shneiderman B (1994) Data structures for dynamic queries: An analytical and experimental evaluation.. In: Proceedings of the workshop on advanced visual interfaces. ACM, NY, pp 1–11CrossRefGoogle Scholar
  7. 7.
    Kim SH, Lu Y, Constantinou G, Shahabi C, Wang G, Zimmermann R (2014) Mediaq: mobile multimedia management system. In: ACM MMSys, pp 224–235Google Scholar
  8. 8.
    Kim Y, Kim J, Yu H (2014) Geotree: using spatial information for georeferenced video search. In: Knowledge-based systems, pp 1–12Google Scholar
  9. 9.
    Lee KC, Lee W-C, Leong HV (2010) Nearest surrounder queries. IEEE TKDE 22(10):1444–1458Google Scholar
  10. 10.
    Li G, Feng J, Xu J (2012) Desks: direction-aware spatial keyword search. In: Proceedings of the 28th IEEE ICDE, pp 474–485Google Scholar
  11. 11.
    Liu X, Shekhar S, Chawla S (2003) Object-based directional query processing in spatial databases. Proc IEEE TKDE 15(2):295–304Google Scholar
  12. 12.
    Lu Y, Shahabi C, Kim SH (2014) An efficient index for large scale geo-tagged video databases. In: Proceedings of the 22nd ACM SIGSPATIAL international conference on GISGoogle Scholar
  13. 13.
    Ma H, Ay SA, Zimmermann R, Kim SH (2013) Large-scale geo-tagged video indexing and queries. GeoInformaticaGoogle Scholar
  14. 14.
    Navarrete T, Blat J (2002) Videogis: segmenting and indexing video based on geographic information. In: Proceedings of the conference on geographic information science, pp 1–9Google Scholar
  15. 15.
    Schneider M, Chen T, Viswanathan G, Yuan W (2012) Cardinal directions between complex regions. ACM TODS 37(2):8:1–8:40CrossRefGoogle Scholar
  16. 16.
    Shekhar S, Liu X, Chawla S (1999) An object model of direction and its implications. Geoinformatica 3(4):357–379CrossRefGoogle Scholar
  17. 17.
    Shen Z, Arslan Ay S, Kim SH, Zimmermann R (2011) Automatic tag generation and ranking for sensor-rich outdoor videos. In: Proceedings of the 19th ACM international conference on multimedia, pp 93–102Google Scholar
  18. 18.
    Tao Y, Papadias D, Sun J (2003) The tpr*-tree: an optimized spatio-temporal access method for predictive queries. In: Proceedings of the 29th international conference on VLDB, vol 29, pp 790–801Google Scholar
  19. 19.
    Theodoridis Y, Papadias D, Stefanakis E (1996) Supporting direction relations in spatial database systems. In: Proceedings of the 7th international symposium on spatial data handling(SDH’96)Google Scholar
  20. 20.
    Toyama K, Logan R, Roseway A (2003) Geographic location tags on digital images. In: Proceedings of the 11th ACM international conference on MM, pp 156–166Google Scholar
  21. 21.
    Young Whang K, Krishnamurthy R (1991) The multilevel grid file - a dynamic hierarchical multidimensional file structure. In: Proceedings international conference on database systems for advanced applications, pp 449–459Google Scholar
  22. 22.
    Zhu Z, Riseman EM, Hanson AR, Schultz H (2005) An efficient method for geo-referenced video mosaicing for environmental monitoring. Mach Vision Appl 16(4):203–216CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Integrated Media Systems CenterUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations