Skip to main content
SpringerLink
Log in

Geo-registration of wide-baseline panoramic image sequences using a digital map reference

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper describes a method of geo-registering a sequence of panoramic images to a digital map by matching pixel information from the images with information on the building footprint contained in a digital map. Recently, images captured at the ground level using a Mobile Mapping System (MMS), such as the panoramic images displayed by Google Street View, have been considered as a valuable resource for three-dimensional (3D) building modeling. However, the wide intervals between these panoramic images, as well as locational and directional error from the related sensors, make it difficult to analyze the image data. This paper demonstrates a formulation method for connecting pixels in panoramic images with information on footprint vertices and building lines contained in a digital map. To allow both pixel and footprint information consistent in 3D space, each panoramic image is tilt-corrected in pre-processing to upright the image using the estimated pitch and roll of a vehicle and removing the pitch and roll effects from the panoramic image pixels. Through the proposed formulation, a single panoramic image can be easily geo-registered with simple user-provided constraints, and adjacent sequential images can then be automatically geo-registered using point feature matching. Experimental results showed a significant reduction in the locational and directional error of sequential panoramic images, and the proposed vanishing point (VP) based validation process was found to successfully detect failure cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Al-Shahri M, Yilmaz A (2014) Line matching in wide-baseline stereo: a top-down approach. IEEE Trans Image Process 23(9):4199–4210

    MathSciNet  Google Scholar 

  2. Al-Tairi ZH, Rahmat RW, Saripan MI, Sulaiman PS (2014) Skin segmentation using YUV and RGB color spaces. J Inf Process Syst 10(2):283–299

    Article  Google Scholar 

  3. Bazin J, Demoncaux C, Vasseur P, Kweon IS (2011) Rotation estimation and vanishing point extraction by omnidirectional vision in urban environment. Int J Robot Res 31(1):63–81

    Article  Google Scholar 

  4. Cham T, Ciptadi A, Tan W, Pham M (2010) Estimating camera pose from a single urban ground-view omnidirectional image and a 2D building outline map, CVPR

  5. Chang S, Chang H, Yen S, Shih TK (2013) Panoramic human structure maintenance based on invariant features of video frames. Human-centric Comput Infor Sci 3(14):1–18

    Google Scholar 

  6. Cipolla R, Robertson D, Tordoff B (2004) Image-based localisation, Proceeding of International Conference on VSMM, pp 22–29

  7. David P (2011) Orientation descriptors for localization in urban environments, International Conference on IROS, pp 494–501

  8. Goferman S, Zelnik-Manor L, Tal A (2010) Context-aware saliency detection, In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2376–2383

  9. Google Earth, http://www.google.com/earth/

  10. Goswami K, Hong G, Kim B (2014) A novel mesh-based moving object detection technique in video sequence. J Converg 4(3):20–24

    Google Scholar 

  11. Hernandez J, Marcotegui B (2009) Morphological segmentation of building façade images, Proceedings of the 16th IEEE international conference on Image processing, pp 3977–3980

  12. Ho Y (2013) Challenging technical issues of 3D video processing. J Converg 4(1):1–6

    Google Scholar 

  13. Hoiem D, Efros AA, Hebert M (2005) Automatic photo pop-up. ACM Trans Graph 24(3):577–584

    Article  Google Scholar 

  14. Huang ZY, He FZ, Cai XT, Zhou ZQ, Liu J, Liang M, Chen X (2011) Efficient random saliency map detection. Sci China, Ser F: Inf Sci 54(6):1207–1217

    Article  Google Scholar 

  15. Khongkraphan K (2014) An efficient color edge detection using the mahalanobis distance. J Inf Process Syst 10(4):589–601

    Article  Google Scholar 

  16. Kim H, Kang Y, Han S (2014) Automatic 3D city modeling using a digital map and panoramic images from a mobile mapping system. Math Probl Eng

  17. Kim D, Trinh H, Jo K (2007) Object recognition of outdoor environment by segmented regions for robot navigation. Lect Notes Comput Sci:1192–1201

  18. Li X, Wu C, Zach C, Lazebnik S, Frahm J (2011) Modeling and recognition of landmark image collections using iconic scene graphs. Int J Comput Vis 95(3):213–239

    Article  Google Scholar 

  19. Liew LH, Lee BY, Wang YC, Cheah W (2014) Aerial images rectification using non-parametric approach. J Converg 4(2):15–22

    Google Scholar 

  20. Liu H, Chen X, Zhang J, Tu Z, 2013, LSGP: Line-SIFT geometric pattern for wide-baseline image matching. ISCAS:721–724

  21. Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  22. Moreno-Noguer F, Lepetit V, Fua P (2009) Accurate non-iterative O(n) solution to the PnP problem. Int J Comput Vis 81(2):155–166

    Article  Google Scholar 

  23. Shahabi C, Kim SH, Nocera L et al (2014) Janus - multi source event detection and collection system for effective surveillance of criminal activity. J Inf Process Syst 10(1):1–22

    Article  Google Scholar 

  24. Taneja A, Ballan L, Pollefeys M (2012) Registration of spherical panoramic images with cadastral 3D Models. 3DIMPVT:479–486

  25. Torii A, Havlena M, Pajdla T (2009) From Google street view to 3D city modeling, ICCV Workshops

  26. Ventura J, Hollerer T (2013) Structure and motion in urban environments using upright panoramas. Virtual Reality 17(2):147–156

    Article  Google Scholar 

  27. Vipparthi SK, Nagar SK (2014) Color directional local quinary patterns for content based indexing and retrieval. Human-centric Comput Inf Sci 4(6):1–13

    Google Scholar 

  28. Von Gioi R, Jakubowicz J, Morel JM, Randall G (2010) LSD: a fast line segment detector with a false detection control. PAMI 32(4):722–732

    Article  Google Scholar 

  29. Wan G, Li S (2011) Automatic facades segmentation using detected lines and vanishing points. CISP 3:1214–1217

    Google Scholar 

  30. Wang G, Tsui H, Hu Z (2005) Reconstruction of structured scenes from two uncalibrated images. Pattern Recogn Lett 26(2):207–220

    Article  Google Scholar 

  31. Xiao J, Fang T, Zhao P, Lhuiller M, Quan L (2009) Image-based street-side city modeling. ACM Trans Graph 28(5):1–12

    Article  Google Scholar 

  32. Zheng X, Zhang X, Guo P (2011) Building modeling from a single image applied in urban reconstruction. VRCAI 11:225–234

    Google Scholar 

Download references

Acknowledgments

This research was supported by the Basic Research Project of Korea Institute of Machinery and Materials (Project Code : NK190C) supported by a grant from Korea National Research Council of Science & Technology, Development of Integration and Automation Technology for Nuclear Plant Life-cycle Management grant funded by the Korea government Ministry of Knowledge Economy (2011 T100200145) and Human Resources Development program(No. 20134030200300) of the Korea Institute of Energy Technology Evaluation and Planning(KETEP) grant funded by the Korea government Ministry of Trade, Industry and Energy.

Author information

Authors and Affiliations

  1. Korea Institute of Machinery & Materials, 171 Jang-Dong, Yuseong-gu, Daejeon, 305-343, Korea

    Hyungki Kim

  2. Korea Advanced Institute of Science and Technology, 291 Daehak-ro, Yuseong-gu, Daejeon, 305-701, Korea

    Soonhung Han

Authors
  1. Hyungki Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Soonhung Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Hyungki Kim or Soonhung Han.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, H., Han, S. Geo-registration of wide-baseline panoramic image sequences using a digital map reference. Multimed Tools Appl 76, 11215–11233 (2017). https://doi.org/10.1007/s11042-016-3298-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3298-1

Keywords

Search

Navigation