Abstract
In this paper we present a semi-automatic 2D-3D local registration pipeline capable of coloring 3D models obtained from 3D scanners by using uncalibrated images. The proposed pipeline exploits the Structure from Motion (SfM) technique in order to reconstruct a sparse representation of the 3D object and obtain the camera parameters from image feature matches. We then coarsely register the reconstructed 3D model to the scanned one through the Scale Iterative Closest Point (SICP) algorithm. SICP provides the global scale, rotation and translation parameters, using minimal manual user intervention. In the final processing stage, a local registration refinement algorithm optimizes the color projection of the aligned photos on the 3D object removing the blurring/ghosting artefacts introduced due to small inaccuracies during the registration. The proposed pipeline is capable of handling real world cases with a range of characteristics from objects with low level geometric features to complex ones.
Chapter PDF
Similar content being viewed by others
References
El-Hakim, S., Beraldin, J. A.: Detailed 3D reconstruction of monuments using multiple techniques. In: Proceedings of the International Workshop on Scanning for Cultural Heritage Recording-Complementing or Replacing Photogrammetry, pp. 58–64 (2002)
Pollefeys, M., Koch, R., Vergauwen, M., Van Gool, L.: Automated reconstruction of 3D scenes from sequences of images. ISPRS Journal of Photogrammetry and Remote Sensing 55(4), 251–267 (2000)
Fitzgibbon, A.W., Zisserman, A.: Automatic camera recovery for closed or open image sequences. In: Burkhardt, H.-J., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1406, pp. 311–326. Springer, Heidelberg (1998)
Bernardini, F., Martin, I.M., Rushmeier, H.: High-quality texture reconstruction from multiple scans. IEEE Transactions on Visualization and Computer Graphics 7(4), 318–332 (2001)
Liu, L., Stamos, I., Yu, G., Wolberg, G., Zokai, S.: Multiview geometry for texture mapping 2d images onto 3d range data. In: 2006 IEEE Computer Society Conference Computer Vision and Pattern Recognition, vol. 2, pp. 2293–2300. IEEE (2006)
Dellepiane, M., Callieri, M., Ponchio, F., Scopigno, R.: Mapping highly detailed colour information on extremely dense 3D models: the case of david’s restoration. In: Computer Graphics Forum, vol. 27(8), pp. 2178–2187. Blackwell Publishing Ltd. (2008)
Liu, L., Stamos, I.: Automatic 3D to 2D registration for the photorealistic rendering of urban scenes. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 137–143. IEEE (2005)
Dellepiane, M., Scopigno, R.: Global refinement of image-to-geometry registration for color projection on 3D models. In: Digital Heritage International Congress, pp. 39–46. The Eurographics Association (2013)
Li, Y., Low, K. L.: Automatic registration of color images to 3d geometry. In: Proceedings of the 2009 Computer Graphics International Conference, pp. 21–28. ACM (2009)
Dellepiane, M., Marroquim, R., Callieri, M., Cignoni, P., Scopigno, R.: Flow-based local optimization for image-to-geometry projection. IEEE Transactions on Visualization and Computer Graphics 18(3), 463–474 (2012)
Eisemann, M., De Decker, B., Magnor, M., Bekaert, P., De Aguiar, E., Ahmed, N., Sellent, A.: Floating textures. In: Computer Graphics Forum, vol. 27(2), pp. 409–418. Blackwell Publishing Ltd. (2008)
Takai, T., Hilton, A., Matsuyama, T.: Harmonised texture mapping. In: Proc. of 3DPVT (2010)
Liu, A., Bullitt, E., Pizer, S. M.: 3D/2D registration using tubular anatomical structures as a basis. In: Proc. Medical Image Computing and Computer-Assisted Intervention, pp. 952–963 (1998)
Musialski, P., Wonka, P., Aliaga, D.G., Wimmer, M., Gool, L., Purgathofer, W.: A survey of urban reconstruction. Computer Graphics Forum 32(6), 146–177 (2013)
Ip, H.H., Yin, L.: Constructing a 3D individualized head model from two orthogonal views. The Visual Computer 12(5), 254–266 (1996)
Lensch, H., Heidrich, W., Seidel, H. P.: Automated texture registration and stitching for real world models. In: Proceedings of the Eighth Pacific Conference on Computer Graphics and Applications, pp. 317–452. IEEE (2000)
Lensch, H., Heidrich, W., Seidel, H.P.: A silhouette-based algorithm for texture registration and stitching. Graphical Models 63(4), 245–262 (2001)
Ikeuchi, K., Oishi, T., Takamatsu, J., Sagawa, R., Nakazawa, A., Kurazume, R., Okamoto, Y.: The great buddha project: Digitally archiving, restoring, and analyzing cultural heritage objects. International Journal of Computer Vision 75(1), 189–208 (2007)
Kurazume, R., Nishino, K., Zhang, Z., Ikeuchi, K.: Simultaneous 2D images and 3D geometric model registration for texture mapping utilizing reflectance attribute. In: Proc. Fifth ACCV, pp. 99–106 (2002)
Corsini, M., Dellepiane, M., Ganovelli, F., Gherardi, R., Fusiello, A., Scopigno, R.: Fully automatic registration of image sets on approximate geometry. International Journal Of Computer Vision 102(1–3), 91–111 (2013)
Zhu, J.H., Zheng, N.N., Yuan, Z.J., Du, S.Y., Ma, L.: Robust scaling iterative closest point algorithm with bidirectional distance measurement. Electronics Letters 46(24), 1604–1605 (2010)
Callieri, M., Cignoni, P., Scopigno, R.: Reconstructing Textured Meshes from Multiple Range RGB Maps. In: VMV, pp. 419–426 (2002)
Callieri, M., Cignoni, P., Corsini, M., Scopigno, R.: Masked photo blending: Mapping dense photographic data set on high-resolution sampled 3D models. Computers & Graphics 32(4), 464–473 (2008)
Dellepiane, M., Callieri, M., Corsini, M., Cignoni, P., Scopigno, R.: Improved color acquisition and mapping on 3d models via flash-based photography. Journal on Computing and Cultural Heritage (JOCCH) 2(4), 9 (2010)
Pal, N.R., Pal, S.K.: A review on image segmentation techniques. Pattern Recognition 26(9), 1277–1294 (1993)
Naik, S.K., Murthy, C.A.: Hue-preserving color image enhancement without gamut problem. IEEE Transactions on Image Processing 12(2), 1591–1598 (2003)
Gallo, A., Muzzupappa, M., Bruno, F.: 3D reconstruction of small sized objects from a sequence of multi-focused images. Journal of Cultural Heritage 15(2), 173–182 (2014)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)
Arya, S., Mount, D.M., Netanyahu, N.S., Silverman, R., Wu, A.Y.: An optimal algorithm for approximate nearest neighbor searching fixed dimensions. Journal of the ACM (JACM) 45(6), 891–923 (1998)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM 24(6), 381–395 (1981)
Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3D. ACM Transactions on Graphics (TOG) 25(3), 835–846 (2006)
Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from internet photo collections. International Journal of Computer Vision 80(2), 189–210 (2008)
Furukawa, Y., Curless, B., Seitz, S. M., Szeliski, R.: Towards internet-scale multi-view stereo. In: 2010 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 1434–1441). IEEE (2010)
Furukawa, Y., Ponce, J.: Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(8), 1362–1376 (2010)
Zhang, Z.: Iterative point matching for registration of free-form curves (1992)
Aiger, D., Mitra, N. J., Cohen-Or, D.: 4-points congruent sets for robust pairwise surface registration. ACM Transactions on Graphics (TOG) 27(3), 85 (2008)
Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., Salesin, D.H.: Synthesizing realistic facial expressions from photographs. In: ACM SIGGRAPH 2006 Courses, p. 19. ACM (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Serna, C.G., Pillay, R., Trémeau, A. (2015). Data Fusion of Objects Using Techniques Such as Laser Scanning, Structured Light and Photogrammetry for Cultural Heritage Applications. In: Trémeau, A., Schettini, R., Tominaga, S. (eds) Computational Color Imaging. CCIW 2015. Lecture Notes in Computer Science(), vol 9016. Springer, Cham. https://doi.org/10.1007/978-3-319-15979-9_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-15979-9_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15978-2
Online ISBN: 978-3-319-15979-9
eBook Packages: Computer ScienceComputer Science (R0)