Key Frame Selection Algorithms for Automatic Generation of Panoramic Images from Crowdsourced Geo-tagged Videos

  • Seon Ho Kim
  • Ying Lu
  • Junyuan Shi
  • Abdullah Alfarrarjeh
  • Cyrus Shahabi
  • Guanfeng Wang
  • Roger Zimmermann
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8470)


Currently, an increasing number of user-generated videos (UGVs) are being collected – a trend that is driven by the ubiquitous availability of smartphones. Additionally, it has become easy to continuously acquire and fuse various sensor data (e.g., geospatial metadata) together with video to create sensor-rich mobile videos. As a result, large repositories of media contents can be automatically geo-tagged at the fine granularity of frames during video recording. Thus, UGVs have great potential to be utilized in various geographic information system (GIS) applications, for example, as source media to automatically generate panoramic images. However, large amounts of crowdsourced media data are currently underutilized because it is very challenging to manage, browse and explore UGVs.

We propose and demonstrate the use of geo-tagged, crowdsourced mobile videos by automatically generating panoramic images from UGVs for web-based geographic information systems. The proposed algorithms leverage data fusion, crowdsourcing and recent advances in media processing to create large scale panoramic environments very quickly, and possibly even on-demand. Our experimental results demonstrate that by using geospatial metadata the proposed algorithms save a significant amount of time in generating panoramas while not sacrificing image quality.


Geo-tagged videos crowdsourcing key frame selection geospatial metadata panorama 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arslan Ay, S., Zimmermann, R., Kim, S.H.: Viewable Scene Modeling for Geospatial Video Search. In: 6th ACM Intl. Conference on Multimedia, pp. 309–318 (2008)Google Scholar
  2. 2.
    Kim, S.H., Lu, Y., Constantinou, G., Shahabi, C., Wang, G., Zimmermann, R.: MediaQ: Mobile Multimedia Management System. In: ACM Multimedia Systems Conference (2014)Google Scholar
  3. 3.
    Kawanishi, T., Yamazawa, K., Iwasa, H., Takemura, H., Yokoya, N.: Generation of High-resolution Stereo Panoramic Images by Omnidirectional Imaging Sensor using Hexagonal Pyramidal Mirrors. In: 14th International Conference on Pattern Recognition, vol.1, pp. 485–489. IEEE (1998)Google Scholar
  4. 4.
    Zhu, Z., Xu, G., Riseman, E.M., Hanson, A.R.: Fast Generation of Dynamic and Multi-resolution 360 Panorama from Video Sequences. In: Int’l Conference on Multimedia Computing and Systems, pp. 400–406. IEEE (1999)Google Scholar
  5. 5.
    Wagner, D., Mulloni, A., Langlotz, T., Schmalstieg, D.: Real-time Panoramic Mapping and Tracking on Mobile Phones. In: Virtual Reality Conference (VR), pp. 211–218. IEEE (2010)Google Scholar
  6. 6.
    Liu, F., Hu, Y.H., Gleicher, M.L.: Discovering panoramas in web videos. In: 16th ACM International Conference on Multimedia, pp. 329–338. ACM (2008)Google Scholar
  7. 7.
    Szeliski, R.: Video Mosaics for Virtual Environments. IEEE Computer Graphics and Applications 16(2), 22–30 (1996)CrossRefGoogle Scholar
  8. 8.
    Szeliski, R., Shum, H.Y.: Creating Full View Panoramic Image Mosaics and Environment Maps. In: 24th Annual Conference on Computer Graphics and Interactive Techniques, pp. 251–258. ACM Press/Addison-Wesley Publishing Co. (1997)Google Scholar
  9. 9.
    Agarwala, A., Agrawala, M., Cohen, M., Salesin, D., Szeliski, R.: Photographing long scenes with multi-viewpoint panoramas. ACM Transactions on Graphics (TOG) 25, 853–861 (2006)CrossRefGoogle Scholar
  10. 10.
    Zheng, J.Y.: Digital route panoramas. IEEE Multimedia 10(3), 57–67 (2003)CrossRefGoogle Scholar
  11. 11.
    van de Laar, V., Aizawa, K., Hatori, M.: Capturing Wide-view Images with Uncalibrated Cameras. In: Electronic Imaging 1999, pp. 1315–1324. International Society for Optics and Photonics (1998)Google Scholar
  12. 12.
    Nielsen, F.: Randomized Adaptive Algorithms for Mosaicing Systems. IEICE Transactions on Information and Systems 83(7), 1386–1394 (2000)Google Scholar
  13. 13.
    Mann, S., Picard, R.W.: Virtual Bellows: Constructing High Quality Stills from Video. In: International Conference on Image Processing (ICIP), vol. 1, pp. 363–367. IEEE (1994)Google Scholar
  14. 14.
    Peleg, S., Herman, J.: Panoramic Mosaics by Manifold Projection. In: International Conference on Computer Vision and Pattern Recognition, pp. 338–343. IEEE (1997)Google Scholar
  15. 15.
    Steedly, D., Pal, C., Szeliski, R.: Efficiently Registering Video into Panoramic Mosaics. In: 10th International Conference on Computer Vision (ICCV), vol. 2, pp. 1300–1307. IEEE (2005)Google Scholar
  16. 16.
    Hsu, C.T., Cheng, T.H., Beuker, R.A., Horng, J.K.: Feature-based Video Mosaic. In: International Conference on Image Processing, vol. 2, pp. 887–890. IEEE (2000)Google Scholar
  17. 17.
    Fadaeieslam, M.J., Fathy, M., Soryani, M.: Key frames selection into panoramic mosaics. In: 7th International Conference on Information, Communications and Signal Processing (ICICS), pp. 1–5. IEEE (2009)Google Scholar
  18. 18.
    Zhang, Y., Ma, H., Zimmermann, R.: Dynamic Multi-video Summarization of Sensor-Rich Videos in Geo-Space. In: Li, S., El Saddik, A., Wang, M., Mei, T., Sebe, N., Yan, S., Hong, R., Gurrin, C. (eds.) MMM 2013, Part I. LNCS, vol. 7732, pp. 380–390. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  19. 19.
    Kazemi, L., Shahabi, C.: GeoCrowd: Enabling Query Answering with Spatial Crowdsourcing. In: ACM SIGSPATIAL GIS, pp. 189–198 (2012)Google Scholar
  20. 20.
    Brown, M., Lowe, D.: AutoStitch: A New Dimension in Automatic Image Stitching (2008)Google Scholar
  21. 21.
    Lourakis, M.I.A., Argyros, A.A.: SBA: A Software Package for Generic Sparse Bundle Adjustment. ACM Transactions on Mathematical Software, 1–30 (2009)Google Scholar
  22. 22.
    Fadaeieslam, M., Soryani, M., Fathy, M.: Efficient Key Frames Selection for Panorama Generation from Video. Journal of Electronic Imaging 20(2), 023015 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Seon Ho Kim
    • 1
  • Ying Lu
    • 1
  • Junyuan Shi
    • 1
  • Abdullah Alfarrarjeh
    • 1
  • Cyrus Shahabi
    • 1
  • Guanfeng Wang
    • 2
  • Roger Zimmermann
    • 2
  1. 1.Integrated Media Systems CenterUniv. of Southern CaliforniaLos AngelesUSA
  2. 2.School of ComputingNational University of SingaporeSingapore

Personalised recommendations