Interactive Virtual Relighting and Remodeling of Real Scenes

  • Céline Loscos
  • Marie-Claude Frasson
  • George Drettakis
  • Bruce Walter
  • Xavier Granier
  • Pierre Poulin
Part of the Eurographics book series (EUROGRAPH)


Lighting design is often tedious due to the required physical manipulation of real light sources and objects. As an alternative, we present an interactive system to virtually modify the lighting and geometry of scenes with both real and synthetic objects, including mixed real/virtual lighting and shadows.

In our method, real scene geometry is first approximately reconstructed from photographs. Additional images are taken from a single viewpoint with a real light in different positions to estimate reflectance. A filtering process is used to compensate for inaccuracies, and per image reflectances are averaged to generate an approximate reflectance image for the given viewpoint, removing shadows in the process. This estimate is used to initialise a global illumination hierarchical radiosity system, representing real-world secondary illumination; the system is optimized for interactive updates. Direct illumination from lights is calculated separately using ray-casting and a table for efficient reuse of data where appropriate.

Our system allows interactive modification of light emission and object positions, all with mixed real/virtual illumination effects. Real objects can also be virtually removed using texture-filling algorithms for reflectance estimation.


Virtual Object Texture Synthesis Reflectance Image Direct Illumination Global Illumination 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    D. R. Baum, H. E. Rushmeier, and J. M. Winget. Improving radiosity solutions through the use of analytically determined form-factors. In Computer Graphics (SIGGRAPH’ 89 Proceedings), volume 23, pages 325–334, July 1989.CrossRefGoogle Scholar
  2. 2.
    P.E. Debevec. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In SIGGRAPH’ 98 Conference Proceedings, Annual Conference Series, pages 189–198, July 1998.Google Scholar
  3. 3.
    P.E. Debevec and J. Malik. Recovering high dynamic range radiance maps from photographs. In SIGGRAPH’ 97 Conference Proceedings, Annual Conference Series, pages 369–378, August 1997.Google Scholar
  4. 4.
    P.E. Debevec, C.J. Taylor, and J. Malik. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In SIGGRAPH’ 96 Conference Proceedings, Annual Conference Series, pages 11–20, July 1996.Google Scholar
  5. 5.
    G. Drettakis, L. Robert, and S. Bougnoux. Interactive common illumination for computer augmented reality. In Rendering Techniques’ 97 (8th Eurographics Workshop on Rendering), pages 45–56. Springer-Verlag, June 1997.Google Scholar
  6. 6.
    G. Drettakis and F. Sillion. Interactive update of global illumination using a line-space hierarchy. In SIGGRAPH’ 97 Conference Proceedings, Annual Conference Series, pages 57–64, August 1997.Google Scholar
  7. 7.
    T. El-Maraghi. An implementation of Heeger and Bergen’s texture analysis/synthesis algorithm with source code. Scholar
  8. 8.
    O. Faugeras. Three-Dimensional Computer Vision — A Geometric Viewpoint. MIT Press, 1993.Google Scholar
  9. 9.
    O. Faugeras, S. Laveau, L. Robert, G. Csurka, and C. Zeller. 3D reconstruction of urban scenes from sequences of images. Tech. report 2572, INRIA Sophia-Antipolis, May 1995.Google Scholar
  10. 10.
    A. Fournier, A.S. Gunawan, and C. Romanzin. Common illumination between real and computer generated scenes. In Proc. of Graphics Interface’ 93, pages 254–262, May 1993.Google Scholar
  11. 11.
    E. A. Haines and J. R. Wallace. Shaft Culling for Efficient Ray-Traced Radiosity. In Photorealistic Rendering in Computer Graphics (Proceedings of the 2nd Eurographics Workshop on Rendering), New York, NY, 1994. Springer-Verlag.Google Scholar
  12. 12.
    P. Hanrahan, D. Salzman, and L. Aupperle. A rapid hierarchical radiosity algorithm. In Computer Graphics (SIGGRAPH’ 91 Proceedings), volume 25, pages 197–206, July 1991.CrossRefGoogle Scholar
  13. 13.
    D.J. Heeger and J.R. Bergen. Pyramid-Based texture analysis/synthesis. In SIGGRAPH’ 95 Conference Proceedings, Annual Conference Series, pages 229–238, August 1995.Google Scholar
  14. 14.
    H. Igehy and L. Pereira. Image replacement through texture synthesis. In Proceedings of the 1997 IEEE International Conference on Image Processing, 1997.Google Scholar
  15. 15.
    F. Leymarie, A. de la Fortelle, J. Koenderink, A. Kappers, M. Stavridi, B. van Ginneken, S. Muller, S. Krake, O. Faugeras, L. Robert, C. Gauclin, S. Laveau, and C. Zeller. Realise: Reconstruction of reality from image sequences. In International Conference on Image Processing, volume 3, pages 651–654, Lausanne (Switzerland), 1996. IEEE Signal Proc. Soc.Google Scholar
  16. 16.
    C. Loscos and G. Drettakis. Interactive relighting of real scenes. Tech. report 0225, INRIA Rhone-Alpes, November 1998.Google Scholar
  17. 17.
    C. Loscos, G. Drettakis, and L. Robert. Interactive modification of real and virtual lights for augmented reality. In SIGGRAPH’ 98 Technical Sketch (Visual Proceedings), July 1998.Google Scholar
  18. 18.
    E. Nakamae, K. Harada, T. Ishizaki, and T. Nishita. A montage method: The overlaying of the computer generated images onto a background photograph. In Computer Graphics (SIGGRAPH’ 86 Proceedings), volume 20, pages 207–214, August 1986.CrossRefGoogle Scholar
  19. 19.
    Photomodeler. Scholar
  20. 20.
    P. Poulin, M. Ouimet, and M.-C. Frasson. Interactively modeling with photogrammetry. In Rendering Techniques’ 98 (9th Eurographics Workshop on Rendering), pages 93–104. Springer-Verlag, June 1998.Google Scholar
  21. 21.
    Y. Sato, M.D. Wheeler, and K. Ikeuchi. Object shape and reflectance modeling from observation. In SIGGRAPH’ 97 Conference Proceedings, Annual Conference Series, pages 379–387, August 1997.Google Scholar
  22. 22.
    E. Shaw. Hierarchical radiosity for dynamic environments. Computer Graphics Forum, 16(2): 107–118, 1997.CrossRefGoogle Scholar
  23. 23.
    F. X. Sillion. A unified hierarchical algorithm for global illumination with scattering volumes and object clusters. IEEE Transactions on Visualization and Computer Graphics, 1(3):240–254, September 1995.CrossRefGoogle Scholar
  24. 24.
    G.J. Ward. The RADIANCE lighting simulation and rendering system. In Proceedings of SIGGRAPH’ 94, Annual Conference Series, pages 459–472, July 1994.Google Scholar
  25. 25.
    H. Weghorst, G. Hooper, and D. P. Greenberg. Improved computational methods for ray tracing. ACM Transactions on Graphics, 3(1):52–69, January 1984.CrossRefGoogle Scholar
  26. 26.
    Y. Yu, P.E. Debevec, J. Malik, and T. Hawkins. Inverse global illumination: Recovering reflectance models of real scenes from photographs. In SIGGRAPH’ 99 (to appear), 1999.Google Scholar
  27. 27.
    Y. Yu and J. Malik. Recovering photometric properties of architectural scenes from photographs. In Computer Graphics (ACM SIGGRAPH’ 98 Proceedings), July 1998.Google Scholar

Copyright information

© Springer-Verlag/Wien 1999

Authors and Affiliations

  • Céline Loscos
    • 1
  • Marie-Claude Frasson
    • 1
    • 2
  • George Drettakis
    • 1
  • Bruce Walter
    • 1
  • Xavier Granier
    • 1
  • Pierre Poulin
    • 2
  1. 1.iMAGIS-GRAVIR/IMAG-INRIAGrenoble, Cedex 9France
  2. 2.Département d’informatique et de recherche opérationnelleUniversité de MontréalCanada

Personalised recommendations