Interactive Common Illumination for Computer Augmented Reality

  • George Drettakis
  • Luc Robert
  • Sylvain Bougnoux
Part of the Eurographics book series (EUROGRAPH)


The advent of computer augmented reality (CAR), in which computer generated objects mix with real video images, has resulted in many interesting new application domains. Providing common illumination between the real and synthetic objects can be very beneficial, since the additional visual cues (shadows, interreflections etc.) are critical to seamless real-synthetic world integration. Building on recent advances in computer graphics and computer vision, we present a new framework to resolving this problem. We address three specific aspects of the common illumination problem for CAR: (a) simplification of camera calibration and modeling of the real scene; (b) efficient update of illumination for moving CG objects and (c) efficient rendering of the merged world. A first working system is presented for a limited sub-problem: a static real scene and camera with moving CG objects. Novel advances in computer vision are used for camera calibration and user-friendly modeling of the real scene, a recent interactive radiosity update algorithm is adapted to provide fast illumination update and finally textured polygons are used for display. This approach allows interactive update rates on mid-range graphics workstations. Our new framework will hopefully lead to CAR systems with interactive common illumination without restrictions on the movement of real or synthetic objects, lights and cameras.


Computer Graphic Virtual Object Camera Calibration Intrinsic Parameter Point Correspondence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    K.B. Atkinson, editor. Close Range Photogrammetry and Machine Vision. Whittles Publishing, 1996.Google Scholar
  2. 2.
    Ronald T. Azuma. A survey of augmented reality. In Presence: Teleoperators and Virtual Environments (to appear) (earlier version in Course Notes #9: Developing Advanced Virtual Reality Applications, ACM SIGGRAPH (LA, 1995), 20–1 to 20–38, 1997. http://www.cs.unc.edufazuma/azuma_AR.html.
  3. 3.
    D. E. Breen, R. T. Whitaker, E. Rose, and M. Tuceryan. Interactive occlusion and automatic object placement for augmented reality. Computer Graphics Forum, 15 (3): C11 - C22, September 1996.CrossRefGoogle Scholar
  4. 4.
    Shenchang Eric Chen and Lance Williams. View interpolation for image synthesis. In James T. Kajiya, editor, Computer Graphics (SIGGRAPH ‘83 Proceedings), volume 27, pages 279–288, August 1993.Google Scholar
  5. 5.
    Michael F. Cohen, Shenchang Eric Chen, John R. Wallace, and Donald P. Greenberg. A progressive refinement approach to fast radiosity image generation. Computer Graphics,22(4):75–84, August 1988. Proceedings SIGGRAPH ‘88 in Atlanta, USA.CrossRefGoogle Scholar
  6. 6.
    P.E. Debevec, C.J. Taylor, and J. Malik. Modeling and rendering architecture from photographs: a hybrid geometry-and image-based approach. In Holly Rushmeier, editor, SIG-GRAPH, pages 11–20, New Orleans, August 1996.Google Scholar
  7. 7.
    Umesh R. Dhond and J.K. Aggarwal. Structure from stereo–a review. IEEE Transactions on Systems, Man, and Cybernetics, 19 (6): 1489–1510, 1989.MathSciNetCrossRefGoogle Scholar
  8. 8.
    George Drettakis and François Sillion. Interactive update of global illumination using a line-space hierarchy. In Turner Whitted, editor, SIGGRAPH 97 Conference Proceedings (Los Angeles, CA), Annual Conference Series. ACM SIGGRAPH, August 1997.Google Scholar
  9. 9.
    Olivier Faugeras. On the evolution of simple curves of the real projective plane. Technical Report 1998, INRIA, 1993.Google Scholar
  10. 10.
    Olivier Faugeras. Three-Dimensional Computer Vision: a Geometric Viewpoint. MIT Press, 1993.Google Scholar
  11. 11.
    Olivier Faugeras and Stéphane Laveau. Representing three-dimensional data as a collection of images and fundamental matrices for image synthesis. In Proceedings of the International Conference on Pattern Recognition,pages 689–691, Jerusalem, Israel, October 1994. Computer Society Press.Google Scholar
  12. 12.
    Olivier Faugeras, Stéphane Laveau, Luc Robert, Gabriella Csurka, Cyril Zeller, Cyrille Gauclin, and Imed Zoghlami. 3-d reconstruction of urban scenes from image sequences. CVGIP: Image Understanding, 1997. To appear.Google Scholar
  13. 13.
    Olivier Faugeras, Tuan Luong, and Steven Maybank. Camera self-calibration: theory and experiments. In G. Sandini, editor, Proc 2nd ECCV, volume 588 of Lecture Notes in Computer Science, pages 321–334, Santa Margherita Ligure, Italy, May 1992. Springer-Verlag.Google Scholar
  14. 14.
    Alain Fournier, Atjeng S. Gunawan, and Chris Romanzin. Common illumination between real and computer generated scenes. In Proceedings Graphics Interface ‘83, pages 254–263. Morgan Kaufmann publishers, 1993.Google Scholar
  15. 15.
    Eric A. Haines. Shaft culling for efficient ray-traced radiosity. In Brunet and Jansen, editors, Photorealistic Rendering in Comp. Graphics, pages 122–138. Springer Verlag, 1993. Proc. 2nd EG Workshop on Rendering (Barcelona, 1991 ).Google Scholar
  16. 16.
    Pat Hanrahan, David Saltzman, and Larry Aupperle. A rapid hierarchical radiosity algorithm. Computer Graphics,25(4):197–206, August 1991. SIGGRAPH ‘81 Las Vegas.CrossRefGoogle Scholar
  17. 17.
    R.I. Hartley. In defence of the 8-point algorithm. In Proceedings of the 5th International Conference on Computer Vision,pages 1064–1070, Boston, MA, June 1995 IEEE Computer Society Press.Google Scholar
  18. 18.
    M. Irani, P. Anandan, and S. Hsu. Mosaic based representations of video sequences and their applications. In Proceedings of the 5th International Conference on Computer Vision,pages 605–611, Boston, MA, June 1995 IEEE Computer Society Press.Google Scholar
  19. 19.
    P. Jancène, F. Neyret, X. Provot, J-P. Tarel, J-M. Vézien, C. Meilhac, and A. Vérroust. Res: computing the interactions between real and virtual objects in video sequences. In 2nd IEEE Workshop on networked Realities, pages 27–40, Boston, Ma (USA), October 1995. Scholar
  20. 20.
    C.P. Jerian and R. Jain. Structure from motion. a critical analysis of methods. IEEE Transactions on Systems, Man, and Cybernetics, 21 (3): 572–587, 1991.MathSciNetCrossRefGoogle Scholar
  21. 21.
    L. McMillan. Acquiring immersive visual environments with an uncalibrated camera. Technical Report TR95–006, University of North Carolina, 1995.Google Scholar
  22. 22.
    L Robert. Camera calibration without feature extraction. Computer Vision, Graphics, and Image Processing, 63 (2): 314–325, March 1995. also INRIA Technical Report 2204.Google Scholar
  23. 23.
    S.M. Seitz and C.R. Dyer. Physically-valid view synthesis by image interpolation. In Proc. IEEE Workshop on Representation of Visual Scenes, pages 18–25, Cambridge, Massachusetts, USA, June 1995.CrossRefGoogle Scholar
  24. 24.
    François Sillion. A unified hierarchical algorithm for global illumination with scattering volumes and object clusters. IEEE Trans. on Vis. and Comp. Graphics, 1 (3), September 1995.Google Scholar
  25. 25.
    Brian Smits, James Arvo, and Donald P. Greenberg. A clustering algorithm for radiosity in complex environments. In Andrew S. Glassner, editor, SIGGRAPH 94 Conference Proceedings (Orlando, FL), Annual Conference Series, pages 435–442. ACM SIGGRAPH, July 1994.Google Scholar
  26. 26.
    Andrei State, Gentaro Hirota, David T. Chen, Bill Garrett, and Mark Livingston. Superior augmented reality registration by integrating landmark tracking and magnetic tracking. In Holly Rushmeier, editor, SIGGRAPH 96 Conference Proceedings (New Orleans, LO), Annual Conference Series, pages 429–438. ACM SIGGRAPH, Addison Wesley, August 1996.Google Scholar
  27. 27.
    Richard Szeliski. Video mosaics for virtual environments. IEEE Computer Graphics and Applications, 16 (2): 22–30, March 1996.CrossRefGoogle Scholar
  28. 28.
    Zhengyou Zhang, Rachid Deriche, Olivier Faugeras, and Quang-Tuan Luong. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence Journal, 78 (1–2): 87–119, 1994.Google Scholar
  29. 29.
    I. Zoghlami, O. Faugeras, and R. Deriche. Using geometric corners to build a 2d mosaic from a set of images. In Proceedings of the International Conference on Computer Vision and Pattern Recognition,Puerto Rico, June 1997. IEEE.Google Scholar

Copyright information

© Springer-Verlag/Wien 1997

Authors and Affiliations

  • George Drettakis
    • 1
  • Luc Robert
    • 2
  • Sylvain Bougnoux
    • 2
  2. 2.INRIA ROBOTVISSophia-Antipolis CedexFrance

Personalised recommendations