Vision-Based Object Registration for Real-Time Image Overlay

  • Michihiro Uenohara
  • Takeo Kanade
Part of the Lecture Notes in Computer Science book series (LNCS, volume 905)


This paper presents computer vision based techniques for object registration, real-time tracking, and image overlay. The capability can be used to superimpose registered images such as those from CT or MRI onto a video image of a patient’s body. Real-time object registration enables an image to be overlaid consistently onto objects even while the object or the viewer is moving. The video image of a patient’s body is used as input for object registration. Reliable real-time object registration at frame rate (30 Hz) is realized by a combination of techniques, including template matching based feature detection, feature correspondence by geometric constraints, and pose calculation of objects from feature positions in the image. Two types of image overlay systems are presented. The first one registers objects in the image and projects preoperative model data onto a raw camera image. The other computes the position of image overlay directly from 2D feature positions without any prior models. With the techniques developed in this paper, interactive video, which transmits images of a patient to the expert and sends them back with some image overlay, can be realized.


Feature Point Reference Image Visual Tracking Interactive Video Cross Ratio 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    E.R. John, L.S. Prichep, J. Fridman and P. Easton, Neurometrics: computer-assisted differential diagnosis of brain disfunction, Science Vol. 239, pp. 162–169 (1988).CrossRefGoogle Scholar
  2. 2.
    L.S. Hibbard, J.S. McGlone, D.W. Davis, R.A. Hawkins, Three-Dimensional Representation and Analysis of Brain Energy Metabolism, Science, Vol. 236, pp. 1641–1646 (1987).CrossRefGoogle Scholar
  3. 3.
    C. Nastar and N. Ayache, Non-Rigid Analysis in Medical Images: a Physically Based Approach, Proc. 13th Int. Conf. on Information Processing in Medical Imaging, Berlin, Germany, pp. 17–32 (1993).Google Scholar
  4. 4.
    W.E.L. Grimson, T. Lozano-Perez, W.M. Wells, G.J. Ettinger, S.J. White, R. Kikinis, An Automatic Registration Method for Frameless Stereotaxy, Image Guided Surgery, and Enhanced Reality Visualization, Proc. CVPR’94, pp. 430–436, Seattle, WA (1994).Google Scholar
  5. 5.
    C. Pelizzari, K. Tan, D. Levin, G. Chen, J. Baiter, Interactive 3D Patient–Image Registration, Proc. 13th Int. Conf. on Information Processing in Medical Imaging, Berlin, Germany, pp. 132–141 (1993).Google Scholar
  6. 6.
    D. Gennery, Tracking known three-dimensional objects, Proc. 2nd Nation. Conf. Artif. Intell., Pittsburgh, pp. 13–17 (1982).Google Scholar
  7. 7.
    D.G. Lowe, Robust Model-Based Motion Tracking Through the Integration of Search and Estimation, Int. J. Computer Vision, Vol. 8, No. 2, pp. 113–122 (1992).CrossRefGoogle Scholar
  8. 8.
    D.G. Lowe, Fitting Parameterized Three-Dimensional Models to Images, IEEE Trans. Patt. Anal. Mach. Intell. Vol. 13, No. 5, pp. 441–450 (1991).MathSciNetCrossRefGoogle Scholar
  9. 9.
    D.B. Gennery, Visual Tracking of Known Three-Dimensional Objects, Int. J. Computer Vision, Vol. 7, No. 3, pp. 243–270 (1992).CrossRefGoogle Scholar
  10. 10.
    M. Turk and A. Pentland, Face Recognition Using Eigenfaces, Proc. CVPR’91, pp.586–591, Maui, U.S.A. (1991).Google Scholar
  11. 11.
    H. Murase and S. Nayar, Parametric Eigenspace Representation for Visual Learning and Recognition, Tech. Rep. CUCS-054–92, Columbia University, NY (1992).Google Scholar
  12. 12.
    S. Yoshimura and T. Kanade, Fast Template Matching Based on the Normalized Correlation by Using Multiresolution Eigenimages, Proc. IROS’94, Munchen, Germany (1994).Google Scholar
  13. 13.
    O. Amidi, Y. Mesaki, T. Kanade, and M. Uenohara, Research on an Autonomous Vision-Guided Helicopter, Proc. RI/SME Fifth World Conf. on Robotics Research, Cambridge, Massachusetts (1994).Google Scholar
  14. 14.
    I. Weiss, Geometric Invariants and Object Recognition, Int. J. Computer Vision, Vol. 10, No. 3, pp. 207–231 (1993).CrossRefGoogle Scholar
  15. 15.
    J. Munday and A. Zisserman, Introduction-Towards a New Framework for Vision. In Geometric Invariance in Machine Vision, MIT Press, Cambridge, MA (1992).Google Scholar
  16. 16.
    H. F. Dunant-Whyte, Uncertain Geometry in Robotics, IEEE J Robotics and Automation, Vol. 4, No. 1, pp. 23–31 (1988).CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • Michihiro Uenohara
    • 1
  • Takeo Kanade
    • 2
  1. 1.Toshiba R&D CenterKawasakiJapan
  2. 2.The Robotics InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations