Abstract
Image-based modeling and rendering has been demonstrated as a cost-effective and efficient approach to computer games and virtual reality applications. The computational model that most image-based techniques are based on is the plenoptic function. Since the original formulation of the plenoptic function does not include illumination, previous image-based applications simply assume that the illumination is fixed. We have proposed anew formulation of the plenoptic function, called the plenoptic illumination function, which explicitly specifies the illumination component. Techniques based on this formulation can be extended to support relighting as well as view interpolation. The core of this framework is compression, and we show how to exploit three types of data correlation, the intra-pixel, the inter-pixel and the inter-channel correlations, in order to achieve a manageable storage size. The proposed coding method outperforms JPEG, JPEG2000 and MPEG.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
E. H. Adelson and J. R. Bergen. “The plenoptic function and the elements of early vision”. Computational Models of Visual Processing, Chapter 1, pp. 3–20, M. S. Landy and J. A. Movshon Eds., MIT Press, 1991.
P. N. Belhumeur and D. J. Kriegman. “What is the Set of Images of an object under all possible lighting conditions”. IEEE Conference on Computer Vision and Pattern Recognition, 1996, sig-11-1994.
B. Cabral, N. Max and R. Springmeyer. “Bidirectional reflection functions from surface bump maps”. Computer Graphics (SIGGRAPH’ 87 Proceedings, vol. 21, no. 4, pp. 273–281, July, 1997.
J. X. Chai, X. Tong, S. C. Chan and H. Y. Shum. “Plenoptic sampling”. Proceedings of SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, pp. 307–318, July, 2000.
M. Charriera, D. S. Cruz and M. Larsson. “JPEG2000, the next millenium compression standard for still images”. Proceedings of the IEEE ICMCS’99, June, 1999.
S. E. Chen. “QuickTime VR-an image-based approach to virtual environment navigation”. Computer Graphics Proceedings. Annual Conference Series, SIGGRAPH’95, pp. 29–38, August, 1995.
R. Courant and D. Hilbert. Methods of Mathematical Physics. Interscience Publisher, Inc., New York, 1953.
S. J. Gortler, R. Grzeszczuk, R. Szeliski and M. F. Cohen. “The lumigraph”. Computer Graphics Proceedings, Annual Conference Series, SIGGRAPH’96, pp. 43–54, August, 1996.
R. M. Gray. Source Coding Theory,Kluwer Academic, 1990.
P. M. Ho, T. T. Wong, K. H. Choy and C. S. Leung. “PCA-based compression for image-based relighting”. Proceedings of IEEE International Conference on Multimedia & Expo 2003 (ICME 2003), Baltimore, USA, July, 2003, to appear.
R. L. Joshi, H. Jafarkani, J. H. Kasner, T. R. Fischer, N. Farvardin, M. W. Marcellin and R. H. Bamberger. “Comparison of different methods of classification in subband coding of images”. IEEE Transaction on Image Processing, vol. 6, pp. 1473–1486, 1997.
J. T. Kajiya. “Anisotropic reflection models”. Computer Graphics (SIGGRAPH’ 85 Proceedings), vol. 19, no. 3, pp. 15–21, July, 1985.
E. P. F. Lafortune, S. C. Foo, K. E. Torrance and D. P. Greenberg. “Non-linear approximation of reflectance functions”. Computer Graphics Proceedings, Annual Conference Series 1997, pp. 117–126, August, 1997.
C. S. Leung, P. M. Lam and T. T. Wong. “Data compression for a massive image data set in IBMR”. Proceedings of Computer Graphics International 2003 (CGI 2003), Tokyo, Japan, July, 2003, to appear.
M. Levoy and P. Hanrahan. “Light field rendering”. Computer Graphics Proceedings, Annual Conference Series, SIGGRAPH’96, pp 31–42, August, 1996.
Z. Lin and T. T. Wong and H. Y. Shum. “Relighting with the reflected irradiance field: representation sampling and reconstruction”. International Journal of Computer Vision, vol. 49, no. 2–3, pp. 229–246, September–October, 2002.
D. Lischinski, F. Tampieri and D. P. Greenberg. “Discontinuity meshing for accurate radiosity”. IEEE Computer Graphics and Applications, vol. 12, no. 6, pp.25–39, Nov, 1992.
L. McMillan and G. Bishop. “Plenoptic modeling: An image-based rendering system”. Computer Graphics Proceedings, Annual Conference Series, SIGGRAPH’95, pp. 39–46, August, 1995.
S. K. Nayar and H. Murase. “Dimensionlity of illumination in appearance matching”, IEEE International Conference on Robotics and Automation, pp. 1326–1332, April, 1996.
J. S. Nimeroff, E. Simoncelli and J. Dorsey. “Efficient re-rendering of naturally illuminated environments”. Fifth Eurographics Workshop on Rendering pp. 359–373, Darmstadt, Germany, June, 1994, sig-11-1994.
K. Nishino, Y. Sato and K. Ikeuchi. “Eigen-texture method: Appearance compression based on 3D model”. Proceedings of IEEE Computer Vision and Pattern Recognition Conference 99, pp. 618–624, June, 1999.
K. R. Rao. Discrete Cosine Transform, Algorithms, Advantages and Applications, Academic Press, 1990.
R.C. Reininger and J.D. Gibson. “Distribution of the two-dimensional DCT coefficients for images”. IEEE Transactions on Communications, vol. 31, pp. 835–839, 1983.
P. Schröder and W. Sweldens. “Spherical wavelets: efficiently representing functions on the sphere”. Annual Conference Series, SIGGRAPH 95 Conference Proceedings, pp. 161–172, August, 1995.
H. Y. Shum and L. W. He. “Rendering with concentric mosaics”. Computer Graphics Proceedings, Annual Conference Series, SIGGRAPH’99, pp. 299–306, August, 1999.
F. X. Sillion, J. R. Arvo, S. H. Westin, Donald P. Greenberg. “A global illumination solution for general reflectance distributions”. Computer Graphics (SIGGRAPH ‘91 Proceedings), vol. 25, no.4, pp. 187–196, July, 1991.
T. T. Wong. “Image-based relighting as the sampling and reconstruction of the plenop-tic illumination function”. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2003), vol. IV, pp. 764–767, Hong Kong, April, 2003.
C. W. Fu, T. T. Wong and P. A. Heng. “Interactive relighting of panoramas”. IEEE Computer Graphics & Applications, vol. 21, no. 2, pp. 32–41, March–April, 2001.
T. T. Wong, C. W. Fu, P. A. Heng and Chi-Sing Leung. “The plenoptic illumination function”. IEEE Transactions on Multimedia, vol. 4, no. 3, pp. 361–371, September, 2001.
T. T. Wong, P. A. Heng, S. H. Or and W. Y. Ng. “Image-based rendering with controllable illumination”. Eighth Eurographics Workshop on Rendering, pp, 13–22, Saint Etienne, France, June, 1997.
T. T. Wong and C. S. Leung. “Compression of illumination-adjustable images”. IEEE Transactions on Circuits and Systems for Video Technology (special issue on Image-based Modeling, Rendering and Animation), 2003, to appear.
T. T. Wong, S. H. Or and C. W. Fu. “Real-time relighting of compressed panoramas”. Graphics Programming Methods, J. Lander Ed., Charles Rivers Media, 2003, to appear.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Kluwer Academic Publishers
About this chapter
Cite this chapter
Wong, TT., Heng, PA. (2004). Image-Based Relighting: Representation and Compression. In: Zhang, D.D., Kamel, M., Baciu, G. (eds) Integrated Image and Graphics Technologies. The International Series in Engineering and Computer Science, vol 762. Springer, Boston, MA. https://doi.org/10.1007/1-4020-7775-0_9
Download citation
DOI: https://doi.org/10.1007/1-4020-7775-0_9
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4020-7774-6
Online ISBN: 978-1-4020-7775-3
eBook Packages: Springer Book Archive