Abstract
This paper describes a method of geo-registering a sequence of panoramic images to a digital map by matching pixel information from the images with information on the building footprint contained in a digital map. Recently, images captured at the ground level using a Mobile Mapping System (MMS), such as the panoramic images displayed by Google Street View, have been considered as a valuable resource for three-dimensional (3D) building modeling. However, the wide intervals between these panoramic images, as well as locational and directional error from the related sensors, make it difficult to analyze the image data. This paper demonstrates a formulation method for connecting pixels in panoramic images with information on footprint vertices and building lines contained in a digital map. To allow both pixel and footprint information consistent in 3D space, each panoramic image is tilt-corrected in pre-processing to upright the image using the estimated pitch and roll of a vehicle and removing the pitch and roll effects from the panoramic image pixels. Through the proposed formulation, a single panoramic image can be easily geo-registered with simple user-provided constraints, and adjacent sequential images can then be automatically geo-registered using point feature matching. Experimental results showed a significant reduction in the locational and directional error of sequential panoramic images, and the proposed vanishing point (VP) based validation process was found to successfully detect failure cases.
Similar content being viewed by others
References
Al-Shahri M, Yilmaz A (2014) Line matching in wide-baseline stereo: a top-down approach. IEEE Trans Image Process 23(9):4199–4210
Al-Tairi ZH, Rahmat RW, Saripan MI, Sulaiman PS (2014) Skin segmentation using YUV and RGB color spaces. J Inf Process Syst 10(2):283–299
Bazin J, Demoncaux C, Vasseur P, Kweon IS (2011) Rotation estimation and vanishing point extraction by omnidirectional vision in urban environment. Int J Robot Res 31(1):63–81
Cham T, Ciptadi A, Tan W, Pham M (2010) Estimating camera pose from a single urban ground-view omnidirectional image and a 2D building outline map, CVPR
Chang S, Chang H, Yen S, Shih TK (2013) Panoramic human structure maintenance based on invariant features of video frames. Human-centric Comput Infor Sci 3(14):1–18
Cipolla R, Robertson D, Tordoff B (2004) Image-based localisation, Proceeding of International Conference on VSMM, pp 22–29
David P (2011) Orientation descriptors for localization in urban environments, International Conference on IROS, pp 494–501
Goferman S, Zelnik-Manor L, Tal A (2010) Context-aware saliency detection, In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2376–2383
Google Earth, http://www.google.com/earth/
Goswami K, Hong G, Kim B (2014) A novel mesh-based moving object detection technique in video sequence. J Converg 4(3):20–24
Hernandez J, Marcotegui B (2009) Morphological segmentation of building façade images, Proceedings of the 16th IEEE international conference on Image processing, pp 3977–3980
Ho Y (2013) Challenging technical issues of 3D video processing. J Converg 4(1):1–6
Hoiem D, Efros AA, Hebert M (2005) Automatic photo pop-up. ACM Trans Graph 24(3):577–584
Huang ZY, He FZ, Cai XT, Zhou ZQ, Liu J, Liang M, Chen X (2011) Efficient random saliency map detection. Sci China, Ser F: Inf Sci 54(6):1207–1217
Khongkraphan K (2014) An efficient color edge detection using the mahalanobis distance. J Inf Process Syst 10(4):589–601
Kim H, Kang Y, Han S (2014) Automatic 3D city modeling using a digital map and panoramic images from a mobile mapping system. Math Probl Eng
Kim D, Trinh H, Jo K (2007) Object recognition of outdoor environment by segmented regions for robot navigation. Lect Notes Comput Sci:1192–1201
Li X, Wu C, Zach C, Lazebnik S, Frahm J (2011) Modeling and recognition of landmark image collections using iconic scene graphs. Int J Comput Vis 95(3):213–239
Liew LH, Lee BY, Wang YC, Cheah W (2014) Aerial images rectification using non-parametric approach. J Converg 4(2):15–22
Liu H, Chen X, Zhang J, Tu Z, 2013, LSGP: Line-SIFT geometric pattern for wide-baseline image matching. ISCAS:721–724
Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110
Moreno-Noguer F, Lepetit V, Fua P (2009) Accurate non-iterative O(n) solution to the PnP problem. Int J Comput Vis 81(2):155–166
Shahabi C, Kim SH, Nocera L et al (2014) Janus - multi source event detection and collection system for effective surveillance of criminal activity. J Inf Process Syst 10(1):1–22
Taneja A, Ballan L, Pollefeys M (2012) Registration of spherical panoramic images with cadastral 3D Models. 3DIMPVT:479–486
Torii A, Havlena M, Pajdla T (2009) From Google street view to 3D city modeling, ICCV Workshops
Ventura J, Hollerer T (2013) Structure and motion in urban environments using upright panoramas. Virtual Reality 17(2):147–156
Vipparthi SK, Nagar SK (2014) Color directional local quinary patterns for content based indexing and retrieval. Human-centric Comput Inf Sci 4(6):1–13
Von Gioi R, Jakubowicz J, Morel JM, Randall G (2010) LSD: a fast line segment detector with a false detection control. PAMI 32(4):722–732
Wan G, Li S (2011) Automatic facades segmentation using detected lines and vanishing points. CISP 3:1214–1217
Wang G, Tsui H, Hu Z (2005) Reconstruction of structured scenes from two uncalibrated images. Pattern Recogn Lett 26(2):207–220
Xiao J, Fang T, Zhao P, Lhuiller M, Quan L (2009) Image-based street-side city modeling. ACM Trans Graph 28(5):1–12
Zheng X, Zhang X, Guo P (2011) Building modeling from a single image applied in urban reconstruction. VRCAI 11:225–234
Acknowledgments
This research was supported by the Basic Research Project of Korea Institute of Machinery and Materials (Project Code : NK190C) supported by a grant from Korea National Research Council of Science & Technology, Development of Integration and Automation Technology for Nuclear Plant Life-cycle Management grant funded by the Korea government Ministry of Knowledge Economy (2011 T100200145) and Human Resources Development program(No. 20134030200300) of the Korea Institute of Energy Technology Evaluation and Planning(KETEP) grant funded by the Korea government Ministry of Trade, Industry and Energy.
Author information
Authors and Affiliations
Corresponding authors
Rights and permissions
About this article
Cite this article
Kim, H., Han, S. Geo-registration of wide-baseline panoramic image sequences using a digital map reference. Multimed Tools Appl 76, 11215–11233 (2017). https://doi.org/10.1007/s11042-016-3298-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-016-3298-1