Interactive Virtual Relighting and Remodeling of Real Scenes
Lighting design is often tedious due to the required physical manipulation of real light sources and objects. As an alternative, we present an interactive system to virtually modify the lighting and geometry of scenes with both real and synthetic objects, including mixed real/virtual lighting and shadows.
In our method, real scene geometry is first approximately reconstructed from photographs. Additional images are taken from a single viewpoint with a real light in different positions to estimate reflectance. A filtering process is used to compensate for inaccuracies, and per image reflectances are averaged to generate an approximate reflectance image for the given viewpoint, removing shadows in the process. This estimate is used to initialise a global illumination hierarchical radiosity system, representing real-world secondary illumination; the system is optimized for interactive updates. Direct illumination from lights is calculated separately using ray-casting and a table for efficient reuse of data where appropriate.
Our system allows interactive modification of light emission and object positions, all with mixed real/virtual illumination effects. Real objects can also be virtually removed using texture-filling algorithms for reflectance estimation.
KeywordsVirtual Object Texture Synthesis Reflectance Image Direct Illumination Global Illumination
Unable to display preview. Download preview PDF.
- 2.P.E. Debevec. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In SIGGRAPH’ 98 Conference Proceedings, Annual Conference Series, pages 189–198, July 1998.Google Scholar
- 3.P.E. Debevec and J. Malik. Recovering high dynamic range radiance maps from photographs. In SIGGRAPH’ 97 Conference Proceedings, Annual Conference Series, pages 369–378, August 1997.Google Scholar
- 4.P.E. Debevec, C.J. Taylor, and J. Malik. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In SIGGRAPH’ 96 Conference Proceedings, Annual Conference Series, pages 11–20, July 1996.Google Scholar
- 5.G. Drettakis, L. Robert, and S. Bougnoux. Interactive common illumination for computer augmented reality. In Rendering Techniques’ 97 (8th Eurographics Workshop on Rendering), pages 45–56. Springer-Verlag, June 1997.Google Scholar
- 6.G. Drettakis and F. Sillion. Interactive update of global illumination using a line-space hierarchy. In SIGGRAPH’ 97 Conference Proceedings, Annual Conference Series, pages 57–64, August 1997.Google Scholar
- 7.T. El-Maraghi. An implementation of Heeger and Bergen’s texture analysis/synthesis algorithm with source code. http://www.cs.toronto.edu/~tem/2522/texture.html.Google Scholar
- 8.O. Faugeras. Three-Dimensional Computer Vision — A Geometric Viewpoint. MIT Press, 1993.Google Scholar
- 9.O. Faugeras, S. Laveau, L. Robert, G. Csurka, and C. Zeller. 3D reconstruction of urban scenes from sequences of images. Tech. report 2572, INRIA Sophia-Antipolis, May 1995.Google Scholar
- 10.A. Fournier, A.S. Gunawan, and C. Romanzin. Common illumination between real and computer generated scenes. In Proc. of Graphics Interface’ 93, pages 254–262, May 1993.Google Scholar
- 11.E. A. Haines and J. R. Wallace. Shaft Culling for Efficient Ray-Traced Radiosity. In Photorealistic Rendering in Computer Graphics (Proceedings of the 2nd Eurographics Workshop on Rendering), New York, NY, 1994. Springer-Verlag.Google Scholar
- 13.D.J. Heeger and J.R. Bergen. Pyramid-Based texture analysis/synthesis. In SIGGRAPH’ 95 Conference Proceedings, Annual Conference Series, pages 229–238, August 1995.Google Scholar
- 14.H. Igehy and L. Pereira. Image replacement through texture synthesis. In Proceedings of the 1997 IEEE International Conference on Image Processing, 1997.Google Scholar
- 15.F. Leymarie, A. de la Fortelle, J. Koenderink, A. Kappers, M. Stavridi, B. van Ginneken, S. Muller, S. Krake, O. Faugeras, L. Robert, C. Gauclin, S. Laveau, and C. Zeller. Realise: Reconstruction of reality from image sequences. In International Conference on Image Processing, volume 3, pages 651–654, Lausanne (Switzerland), 1996. IEEE Signal Proc. Soc.Google Scholar
- 16.C. Loscos and G. Drettakis. Interactive relighting of real scenes. Tech. report 0225, INRIA Rhone-Alpes, November 1998.Google Scholar
- 17.C. Loscos, G. Drettakis, and L. Robert. Interactive modification of real and virtual lights for augmented reality. In SIGGRAPH’ 98 Technical Sketch (Visual Proceedings), July 1998.Google Scholar
- 19.Photomodeler. http://www.photomodeler.com.Google Scholar
- 20.P. Poulin, M. Ouimet, and M.-C. Frasson. Interactively modeling with photogrammetry. In Rendering Techniques’ 98 (9th Eurographics Workshop on Rendering), pages 93–104. Springer-Verlag, June 1998.Google Scholar
- 21.Y. Sato, M.D. Wheeler, and K. Ikeuchi. Object shape and reflectance modeling from observation. In SIGGRAPH’ 97 Conference Proceedings, Annual Conference Series, pages 379–387, August 1997.Google Scholar
- 24.G.J. Ward. The RADIANCE lighting simulation and rendering system. In Proceedings of SIGGRAPH’ 94, Annual Conference Series, pages 459–472, July 1994.Google Scholar
- 26.Y. Yu, P.E. Debevec, J. Malik, and T. Hawkins. Inverse global illumination: Recovering reflectance models of real scenes from photographs. In SIGGRAPH’ 99 (to appear), 1999.Google Scholar
- 27.Y. Yu and J. Malik. Recovering photometric properties of architectural scenes from photographs. In Computer Graphics (ACM SIGGRAPH’ 98 Proceedings), July 1998.Google Scholar