Advertisement

Depth Map Generation for 3-D TV: Importance of Edge and Boundary Information

  • Wa James Tam
  • Filippo Speranza
  • Liang Zhang
Chapter

Introduction

As an ongoing and natural development, it is reasonable to regard stereoscopic three-dimensional television (3-D TV) as the next desirable milepost beyond high-definition television (HDTV). This direction of development is evidenced by the increasing number of scientific and engineering conferences held each year on 3-D TV and related technologies. At the same time, we see that industry, academia, and standard organizations have launched major research and standardization initiatives, such as the 3-D Consortium in Japan [1], the European IST research project ATTEST [2, 3] and the JVT/MVC (MPEG-3-D) standardization effort [4, 5].

Three-dimensional television is desirable not only because it can enhance perceived depth, but also because it promises to increase the feeling of presence and realism in television viewing [6, 7]. These attributes can heighten the entertainment value of the television medium, thereby attracting larger audiences and increasing revenues. Despite...

Keywords

Depth Information Object Boundary Stereoscopic Image Disparity Estimation Stereoscopic Depth 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

The authors wish to thank Ron Renaud for his technical assistance in setting up the experimental apparatus and for helping with the data collection for the studies. We also would like to thank co-op students Guillaume Alain, Jorge Ferreira, Sharjeel Tariq, Anthony Soung Yee and Jason Chan Wai for developing software tools and algorithms for the experiments. Results of the first study were presented at the conference on Three-Dimensional Displays and Applications, San Jose, CA., USA, January 2005, and those for the second were presented at the conference on Three-Dimensional Television, Video and Display, Philadelphia, PA., USA, October 2005. Thanks are also due to Carlos Vázquez, Xianbin Wang, and André Vincent for their suggestions and comments on an earlier version of the manuscript. Finally, “Interview" was provided by HHI (Germany) while the images, “Aqua," “Meal," “Red Leaves," and “Tulips," were produced by NHK (Japan). “Cones" is a cropped image of an original that was obtained from the Stereo Vision Research Page maintained by D. Scharstein and R. Szeliski at http://http://cat.middlebury.edu/stereo (accessed June 2007).

References

  1. [1]
    3D Consortium. [June 20, 2007 available on-line http://www.3-Dc.gr.jp/ english/]
  2. [2]
    European IST-2001-34396 project: ATTEST, 2002–2004. [June 20, 2007 available on-line http://www.extra.research.philips.com/euprojects/attest/]
  3. [3]
    C. Fehn, “A 3D-TV system based on video plus depth information,” 37th Asilomar Conference on Signals, Systems, and Computers, Vol. 2, pp. 1529–1533, 2003.Google Scholar
  4. [4]
    ISO/IEC JTC1/SC29/WG11, “Information technology – MPEG Video Technologies – Part 3: Representation of auxiliary video and supplemental information," ISO/IEC FDIS 23002-3, January 19, 2007.Google Scholar
  5. [5]
    A. Smolic & H. Kimata, “Report on 3DAV exploration,” ISO/IEC JTC1/SC29/WG11 Doc N5878, July 2003.Google Scholar
  6. [6]
    S. Yano & I. Yuyama, “Stereoscopic HDTV: Experimental system and psychological effects,” Journal of the SMPTE, Vol. 100, pp.14–18, 1991.Google Scholar
  7. [7]
    W. A. IJsselsteijn, H. de Ridder, R. Hamberg, D. Bouwhuis, & J. Freeman, “Perceived depth and the feeling of presence in 3DTV,” Displays, Vol. 18, pp. 207–214, 1998.CrossRefGoogle Scholar
  8. [8]
    J. Ouchi, H. Kamei, K. Kikuta, & Y. Takaki, “Development of 128-directional 3D display system," Three-Dimensional TV, Video, and Display V, Vol. 6392, pp. 63920I, 2006.Google Scholar
  9. [9]
    P. Merkle, K. Müller, A. Smolic, & T. Wiegand, “Efficient compression of multi-view video exploiting inter-view dependencies based on H.264/MPEG4-AVC," IEEE International Conference on Multimedia and Exposition (ICME2006), pp. 1717–1720, 2006.Google Scholar
  10. [10]
    K. T. Kim, M. Siegel, & J. Y. Son, “Synthesis of a high-resolution 3D stereoscopic image pair from a high-resolution monoscopic image and a low-resolution depth map," Stereoscopic Displays and Applications IX, Vol. 3295A, pp. 76–86, 1998.Google Scholar
  11. [11]
    J. Flack, P. Harman, & S. Fox, “Low bandwidth stereoscopic image encoding and transmission, ” Stereoscopic Displays and Virtual Reality Systems X, Vol. 5006, pp. 206–214, 2003.Google Scholar
  12. [12]
    C. Fehn, “A 3D-TV approach using depth-image-based rendering (DIBR),” Visualization, Imaging, and Image Processing (VIIP’03), pp. 482–487, 2003.Google Scholar
  13. [13]
    C. Fehn, “Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV", Stereoscopic Displays and Virtual Reality Systems XI, Vol. 5291, pp. 93–104, 2004.Google Scholar
  14. [14]
    L. Zhang & W. J. Tam, “Stereoscopic image generation based on depth images for 3D TV,” IEEE Transactions on Broadcasting, Vol. 51, pp. 191–199, 2005.CrossRefGoogle Scholar
  15. [15]
    L. McMillan, “An image-based approach to three-dimensional computer graphics," Ph.D. thesis, University of North Carolina at Chapel Hill, 1997. [June 20, 2007 available on-line http://ftp://ftp.cs.unc.edu/pub/publications/ techreports/97-013.pdf]
  16. [16]
    W. R. Mark, “Post-rendering 3D image warping: Visibility, reconstruction and performance for depth-image warping," Ph.D. thesis, University of North Carolina at Chapel Hill, 1999. [June 20, 2007 available on-line: http://www-csl.csres.utexas.edu/users/billmark/papers/dissertation/TR99-022.pdf]
  17. [17]
    L. Zhang, W. J. Tam, & D. Wang, “Stereoscopic image generation based on depth images,” IEEE Conference on Image Processing, Vol. 5, pp. 2993–2996, 2004.Google Scholar
  18. [18]
    C. Vázquez, W. J. Tam, & F. Speranza, “Stereoscopic imaging: Filling disoccluded areas in image-based rendering,” Three-Dimensional TV, Video, and Display (ITCOM2006), Vol. 6392, pp. 0D1–0D12, 2006.Google Scholar
  19. [19]
    W. J. Tam, G. Alain, L. Zhang, T. Martin, & R. Renaud, “Smoothing depth maps for improved stereoscopic image quality," Three-Dimensional TV, Video and Display III (ITCOM'04), Vol. 5599, pp.162–172, 2004.Google Scholar
  20. [20]
    W. J. Tam & L. Zhang, “Non-uniform smoothing of depth maps before image-based rendering," Three-Dimensional TV, Video and Display III (ITCOM'04), Vol. 5599, pp. 173–183, 2004.Google Scholar
  21. [21]
    G. Iddan & G. Yahav, “3D imaging in the studio," Videometrics and Optical Methods for 3-D Shape Measurement, Vol. 4298, pp. 48–55, 2001.Google Scholar
  22. [22]
    M. Kawakita, K. Iizuka, H. Nakamura, I. Mizuno, T. Kurita, T. Aida, Y. Yamanouchi, H. Mitsumine, T. Fukaya, H.Kikuchi, & F. Sato, “High-definition real-time depth-mapping TV camera: HDTV Axi-Vision camera," Optics Express, Vol. 12, pp. 2781–2794, 2004.ADSCrossRefGoogle Scholar
  23. [23]
    D. Scharstein & R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," International Journal of Computer Vision, Vol. 47, pp.7–42, 2002.MATHCrossRefGoogle Scholar
  24. [24]
    L. Zhang, W. J. Tam, G. Um, F. Speranza, N. Hur, & A. Vincent, “Virtual view generation based on multiple images,” IEEE Conference on Multimedia & Expo (ICME), Beijing, July, 2007.Google Scholar
  25. [25]
    M. W. Eysenck, “Visual perception." In Psychology: An international perspective, Chapter 7, pp.218–259, London: Psychology Press, 2004.Google Scholar
  26. [26]
    S. Battiato, S. Curti, E. Scordato, M. Tortora, & M. La Cascia, “Depth map generation by image classification,” Three-Dimensional Image Capture and Applications VI, Vol. 5302, pp. 95–104, 2004.Google Scholar
  27. [27]
    S. Battiato, A. Capra, S. Curti, & M. La Cascia, “3-D stereoscopic image pairs by depth-map generation," Second International Symposium on 3-D Data Processing, Visualization and Transmission, pp. 124–131, 2004.Google Scholar
  28. [28]
    L. Zhang, B. Lawrence, D. Wang, & A. Vincent, “Comparison study on feature matching and block matching for automatic 2-D to 3-D video conversion,” The 2nd IEE European Conference on Visual Media Production, pp. 122–129, 2005.Google Scholar
  29. [29]
    J. Ens & P. Lawrence, “An investigation of methods for determining depth from focus," IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 15, pp. 97–108, 1993.CrossRefGoogle Scholar
  30. [30]
    N. Asada, H. Fujiwara, & T. Matsuyama, “Edge and depth from focus," The 1st Asian Conference on Computer Vision, pp. 83–86, 1993.Google Scholar
  31. [31]
    A. P. Pentland, “Depth of scene from depth of field," The Image Understanding Workshop, pp. 253–259, 1982.Google Scholar
  32. [32]
    G. J. Mitchison & S. P. McKee, “Interpolation in stereoscopic matching,” Nature, Vol. 315, pp. 402–404, 1985.ADSCrossRefGoogle Scholar
  33. [33]
    S. M. Wurger & M. S. Landy, “Depth interpolation with sparse disparity cues,” Perception, Vol.18, pp. 39–54, 1989.CrossRefGoogle Scholar
  34. [34]
    J. H. Elder & S. W. Zucker, “Local scale control for edge detection and blur estimation," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, pp. 699–716, 1998.CrossRefGoogle Scholar
  35. [35]
    L. B. Stelmach &W. J. Tam, “Stereoscopic image coding: Effect of disparate image-quality in left- and right-eye views,” Signal Processing: Image Communication, Special Issue on 3-D Video Technology, Vol.14, pp.111–117, 1998.Google Scholar
  36. [36]
    W. J. Tam, L. B. Stelmach, D. Meegan, & A.Vincent, “Bandwidth reduction for stereoscopic video signals,” Proceedings of the SPIE: Stereoscopic Displays and Virtual Reality Systems VII, Vol. 3957, pp. 33–40, 2000.ADSGoogle Scholar
  37. [37]
    W. J. Tam, A. Soung Yee, J. Ferreira, S. Tariq, & F. Speranza, “Stereoscopic image rendering based on depth maps created from blur and edge information," Stereoscopic Displays and Applications XII, Vol. 5664, pp.104–115, 2005.Google Scholar
  38. [38]
    ITU-R Recommendation BT.500-11, “Methodology for the subjective assessment of the quality of television pictures," 2005.Google Scholar
  39. [39]
    D. Scharstein & R. Szeliski, “High-accuracy stereo depth maps using structured light,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’03), Vol. 1, pp. 195–202, 2003.Google Scholar
  40. [40]
    W. J. Tam, F. Speranza, L. Zhang, R. Renaud, J. Chan, & C. Vazquez, “Depth image based rendering for multiview stereoscopic displays: Role of information at object boundaries," Three-Dimensional TV, Video, and Display IV, Vol. 6016, p. 97–107, 2005.Google Scholar
  41. [41]
    M. Ichikawa, S. Saida, A. Osa, & K. Munechika, “Integration of binocular disparity and monocular cues at near threshold level," Vision Research, Vol. 43, pp. 2439–2449, 2003.CrossRefGoogle Scholar
  42. [42]
    E. B. Johnston, B. G. Cumming, & A. J. Parker, “Integration of depth modules: stereopsis and texture," Vision Research, Vol. 33, pp. 813–826, 1993.CrossRefGoogle Scholar
  43. [43]
    S. Grossberg & E. Mingolla, “Neural dynamics of form perception: Boundary completion, illusory figures, and neon color spreading," Psychological Review, Vol. 92, pp. 173–211, 1985.CrossRefGoogle Scholar
  44. [44]
    P. J. Kellman & T. F. Shipley, “A theory of visual interpolation in object perception," Cognitive Psychology, Vol. 23, pp. 141–221, 1991.CrossRefGoogle Scholar
  45. [45]
    C. Yin, P. J. Kellman, & T. F. Shipley, “Surface completion complements boundary interpolation in the visual integration of partly occluded objects," Perception, Vol. 26, pp. 1459–1479, 1997.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Wa James Tam
    • 1
  • Filippo Speranza
  • Liang Zhang
  1. 1.Communications Research Centre CanadaOttawaCanada K2H 8S2

Personalised recommendations