Simultaneous Registration of 2D Images onto 3D Models for Texture Mapping

  • Ryo Ohkubo
  • Ryo Kurazume
  • Katsushi Ikeuchi

Recently, creation of realistic 3D contents through sensing the real world has become fundamental for many applications. To enhance 3D geometric models obtained through laser range scanners with their textures reconstructed from several photographic 2D images taken from various view points, it is necessary to determine the camera position and orientation relative to the 3D models for each of the images.

In this chapter, a registration method is proposed, which automatically and simultaneously aligns multiple 2D images onto a 3D model. For each iteration process, correspondences between 2D edge pixels and 3D edge points are automatically searched and updated. Besides these 2D-3D edge correspondences, 2D-2D edge correspondences on 3D surface model are also considered simultaneously for global optimization among all the images. Errors are minimized by using conjugate gradient search, utilizing M-estimator for robustness. From texture mapped objects, the usefulness of the proposed simultaneous registration method is shown. Also, it is applied to the creation of digital cultural assets.


Registration Method Camera Parameter Texture Mapping Camera Coordinate System Neighboring Image 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    P. K. Allen, I. Stamos, A. Troccoli, B. Smith, M. Leordeanu, and Y. C. Hsu. 3D modeling of historic sites using range and image data. submitted to the International Conference of Robotics and Automation, 2003.Google Scholar
  2. [2]
    P. Besl and N. McKay. A method for registration of 3-D shapes. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 14, No. 2, pp. 239-256, February 1992.CrossRefGoogle Scholar
  3. [3]
    J. Canny. A computational approach to edge detection. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 8, No. 6, pp. 679-698, 1986.CrossRefGoogle Scholar
  4. [4]
    Y. Chen and G. Medioni. Object modeling by registration of multiple range images. Image and Vision Computing, Vol. 10, No. 3, pp. 145-155, 1992.CrossRefGoogle Scholar
  5. [5]
    P. E. Debevec. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. Computer Graphics, pp. 189-198, 1998. Proc. of SIGGRAPH ‘98 (July 1998, Orlando, Florida).Google Scholar
  6. [6]
    P. E. Debevec. A tutorial on image-based lighting. IEEE Computer Graphics and Applications, pp. 26-34, 2002.Google Scholar
  7. [7]
    P. E. Debevec and Jitendra Malik. Recovering high dynamic range radiance maps from photographs. Computer Graphics, pp. 369-378, 1997. Proc. of SIGGRAPH ‘97 (August 1997, Los Angeles, California).Google Scholar
  8. [8]
    P. E. Debevec, C. J. Taylor, and J. Malik. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. Computer Graphics, pp. 11-20, 1996. In Proc. of SIGGRAPH ‘96 (August 1996, New Orleans, Louisiana).Google Scholar
  9. [9]
    K. Deguchi and T. Okatani. Calibration of multi-view cameras for 3D scene understanding (in japanese). CVIM, January 2002.Google Scholar
  10. [10]
    O. Faugeras. Three-Dimensional Computer Vision: A Geometric View-point. The MIT Press, 1993.Google Scholar
  11. [11]
    T. Igarashi, S. Matsuoka, and H. Tanaka. Teddy: A sketching interface for 3D freeform design. Computer Graphics, pp. 409-416, 1999. In Proc. of SIGGRAPH ‘99 (Los Angels, 1999).Google Scholar
  12. [12]
    B. Klaus and P. Horn. Robot Vision. The MIT Press, McGraw-Hill Book Company, 1986.Google Scholar
  13. [13]
    R. Kurazume, K. Nishino, Z. Zhang, and K. Ikeuchi. Simultaneous 2D images and 3D geometric model registration for texture mapping utilizing reflectance attribute. In Proc. 5th Asian Conf. on Computer Vision, January 2002.Google Scholar
  14. [14]
    R. Kurazume, M. D. Wheeler, and K. Ikeuchi. Mapping textures on 3D geometric model using reflectance image. Data Fusion Workshop in IEEE Int. Conf. on Robotics and Automation, May 2001.Google Scholar
  15. [15]
    S. Lavallée and R. Szeliski. Recovering the position and orientation of free-form objects from image contours using 3D distance maps. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 17, No. 4, pp. 378-390, April 1995.CrossRefGoogle Scholar
  16. [16]
    H. P. A. Lensch, W. Heidrich, and H. Seidel. Automated texture registration and stitching for real world models. In Proc. Pacific Graphics ’00, pp. 317-326, October 2000.Google Scholar
  17. [17]
    W. E. Lorensen and H. E. Cline. Marching cubes: A high resolution 3D surface construction algorithm. Computer Graphics, Vol. 21, No. 4, pp. 163-169, July 1987. In Proc. SIGGRAPH ‘87.CrossRefGoogle Scholar
  18. [18]
    D. Miyazaki, T. Ooishi, T. Nishikawa, R. Sagawa, K. Nishino, T. Tomomatsu, Y. Takase, and K. Ikeuchi. The great buddha project: Modelling cultural heritage through observation. In Proc. 6th Inter. Conf. on Virtual Systems and MultiMedia (VSMM 2000), pp. 138-145, 2000.Google Scholar
  19. [19]
    S. K. Nayar and T. Mitsunaga. High dynamic range imaging: Spatially varying pixel exposures. In Proc. IEEE Inter. Conf. on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, June 2000.Google Scholar
  20. [20]
    P. J. Neugebauer and K. Klein. Texturing 3D models of real world objects from multiple unregistered photograpihic views. In Proc. Eurographics ’99, pp. 245-256, Milan, September 1999.Google Scholar
  21. [21]
    K. Nishino and K. Ikeuchi. Robust simultaneous registration of multiple range images. In Proc. 5th Asian Conf. on Computer Vision, pp. 454-461, Melbourne, Australia, January 2002.Google Scholar
  22. [22]
    K. Nishino, Y. Sato, and K. Ikeuchi. Eigen-texture method: Appearance compression and synthesis based on a 3D model. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No. 11, pp. 1257-1265, 2001.CrossRefGoogle Scholar
  23. [23]
    C. Rocchini, P. Cignoni, and C. Montani. Multiple textures stitching and blending on 3D objects. In 10th Eurographics Workshop on Rendering, pp. 127-138, June 1999.Google Scholar
  24. [24]
    S. Rusinkiewicz and M. Levoy. Efficient variants of the ICP algorithm. 3DIM, 2001.Google Scholar
  25. [25]
    R. Sagawa, K. Nishino, and K. Ikeuchi. Robust and adaptive integration of multiple range images with photometric attributes. In Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, pp. 172-179, December 2001.Google Scholar
  26. [26]
    R. Sagawa, T. Oishi, A. Nakazawa, R. Kurazume, and K. Ikeuchi. Iterative refinement of range images with anisotropic error distribution. In Proc. of 2002 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 79-85, October 2002.Google Scholar
  27. [27]
    I. Sato, Y. Sato, and K. Ikeuchi. Acquiring a radiance distribution to superimpose virtual objects onto a real scene. IEEE Trans. Visualization and Computer Graphics, Vol. 5, No. 1, pp. 1-12, 1999.CrossRefGoogle Scholar
  28. [28]
    I. Sato, Y. Sato, and K. Ikeuchi. Illumination distribution from brightness in shadows: adaptive estimation of illumination distribution with unknown reflectance properties in shadow regions. In Proc. IEEE Inter. Conf. Computer Vision, pp. 875-882, September 1999.Google Scholar
  29. [29]
    Y. Sato, M. D. Wheeler, and K. Ikeuchi. Object shape and reflectance modeling from observation. Computer Graphics, pp. 379-387, August 1997. In Proc. SIGGRAPH ‘97.Google Scholar
  30. [30]
    J. Shi and C. Tomasi. Good features to track. In Proc. IEEE Inter. Conf. on Computer Vision and Pattern Recognition, pp. 593-600, 6 1994.Google Scholar
  31. [31]
    I. Stamos and P. K. Allen. 3-D model construction using range and image data. In Proc. IEEE Inter. Conf. on Computer Vision and Pattern Recognition, Vol. 1, pp. 531-536, South Carolina, June 2000.Google Scholar
  32. [32]
    R. Tsai. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf tv cameras and lenses. IEEE Journal of Robotics and Automation, Vol. 3, No. 4, pp. 323-344, August 1987.CrossRefGoogle Scholar
  33. [33]
    M. D. Wheeler. Automatic Modeling and Localization for Object Recognition. PhD thesis, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, October 1996.Google Scholar
  34. [34]
    R. C. Zeleznik, K. P. Herndon, and J. F. Hughes. Sketch: An interface for sketching 3D scenes. Computer Graphics, pp. 163-170, 1996. In Proc. of SIGGRAPH ‘96 (New Orleans,1996).Google Scholar
  35. [35]
    Z. Zhang. Determining the epipolar geometry and its uncertainty: A review. International Journal of Computer Vision, 1997.Google Scholar
  36. [36]
    Z. Zhang. A flexible new technique for camera calibration. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 22, No. 11, pp. 1330-1334,2000.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Ryo Ohkubo
  • Ryo Kurazume
  • Katsushi Ikeuchi
    • 1
  1. 1.Institute of Industrial ScienceThe University of TokyoMeguro-kuJapan

Personalised recommendations