Registration Methods for Harmonious Integration of Real World and Computer Generated Objects

  • Gilles Simon
  • Vincent Lepetit
  • Marie-Odile Berger
Part of the NATO Science Series book series (ASHT, volume 84)


We focus in this chapter on the problem of adding computer-generated objects in video sequences. A two-stage robust statistical method is used for computing the pose from model-image correspondences of tracked curves. This method is able to give a correct estimate of the pose even when tracking errors occur. However, if we want to add virtual objects in a scene area which does not contain (or contains few) model features, the reprojection error in this area is likely to be large. In order to improve the accuracy of the viewpoint, we use 2D keypoints that can be easily matched in two consecutive images. As the relationship between two matched points is a function of the camera motion, the viewpoint can be improved by minimizing a cost function which encompasses the reprojection error as well as the matching error between two frames. The reliability of the system is shown on the encrustation of a virtual car in a sequence of the Stanislas square.

The interested reader can look at the video sequences of our results1.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    M.-O. Berger. How to track efficiently piecewise curved contours with a view to reconstructing 3D objects. In Proceedings of the 12th International Conference on Pattern Recognition, Jerusalem (Israel), volume 1, pages 32–36, 1994.CrossRefGoogle Scholar
  2. [2]
    M.-O. Berger. Resolving occlusion in augmented reality: a contour-based approach without 3D reconstruction. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, PR (USA), pages 91–96, June 1997.Google Scholar
  3. [3]
    M.-O. Berger, C. Chevrier, and G. Simon. Compositing computer and video image sequences: Robust algorithms for the reconstruction of the camera parameters. Computer Graphics Forum, Conference Issue Eurographics’96, Poitiers, France, 15(3):23–32, August 1996.Google Scholar
  4. [4]
    D. Dementhon and L. Davis. Model based object pose in 25 lines of code. International Journal of Computer Vision, 15:123–141, 1995.CrossRefGoogle Scholar
  5. [5]
    G. Ertl, H. Müller-Seelich, and B. Tabatabai. MOVE-X: a system for combining video films and computer animation. In Eurographics, pages 305–313, 1991.Google Scholar
  6. [6]
    O. Faugeras. Three-Dimensional Computer Vision: A Geometric Viewpoint. Artificial Intelligence. MIT Press, 1993.Google Scholar
  7. [7]
    O. D. Faugeras and G. Toscani. The Calibration Problem for Stereo. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL (USA), pages 15–20, 1986.Google Scholar
  8. [8]
    R. M. Haralick, H. Joo, C. N. Lee, X. Zhuang, V.G. Vaidya, and M. B. Kim. Pose estimation from corresponding point data. IEEE Transactions on Systems, Man, and Cybernetics, 19(6), 1989.Google Scholar
  9. [9]
    C. Harris and M. Stephens. A combined corner and edge detector. In Proceedings of 4th Alvey Conference, Cambridge, August 1988.Google Scholar
  10. [10]
    R. Kumar and A. Hanson. Robust methods for estimating pose and a sensitivity analysis. CVGIP: Image Understanding, 60(3):313–342, 1994.CrossRefGoogle Scholar
  11. [11]
    Q.-T. Luong, R. Deriche, O. Faugeras, and T. Papadopoulo. On determining the fundamental matrix: Analysis of different methods and experimental results. Technical Report 1894, INRIA, 1993.Google Scholar
  12. [12]
    W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling. Numerical Recipes in C, The Art of Scientific Computing. Cambridge University Press, 1988.zbMATHGoogle Scholar
  13. [13]
    C. Schmid and R. Mohr. Local grayvalue invariants for image retrieval. IEEE Transactions on RAMI, 19(5):530–535, August 1997.Google Scholar
  14. [14]
    G. Simon and M.-O. Berger. A two-stage robust statistical method for temporal registration from features of various type. In Proceedings of 6th International Conference on Computer Vision, Bombay (India), pages 261–266, January 1998.Google Scholar
  15. [15]
    A. State, G. Hirota, D. Chen, W. Garett, and M. Livingston. Superior augmented reality registration by integrating landmark tracking and magnetic tracking. In Computer Graphics (Proceedings Siggraph New Orleans), pages 429–438, 1996.Google Scholar
  16. [16]
    C. Tomasi and T. Kanade. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9(2):137–154, 1992.CrossRefGoogle Scholar
  17. [17]
    M. Uenohara and T. Kanade. Vision based object registration for real time image overlay. Journal of Computers in Biology and Medicine, 25(2):249–260, 1996.CrossRefGoogle Scholar
  18. [18]
    Z. Zhang, R. Deriche, O. Faugeras, and Q. Luong. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence, 78:87–119, October 1995.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2000

Authors and Affiliations

  • Gilles Simon
  • Vincent Lepetit
  • Marie-Odile Berger

There are no affiliations available

Personalised recommendations