Abstract
In this paper, we present a system to redistribute computational power in plane sweeping. Plane sweeping allows the generation of novel viewpoints of a scene by testing different depth hypotheses across input cameras. Typical plane sweeping approaches incorporate a uniform depth plane distribution to investigate different depth hypotheses to generate a depth map. When the scene consists of a sparse number of objects, some depth hypotheses do not contain objects and can cause noise and wasted computational power. Therefore, we propose a method to adapt the plane distribution to increase the quality of the depth map around objects and to reduce computational power waste by reducing the number of planes in empty spaces in the scene. First, we generate the cumulative histogram of the depth map of the scene. This depth map can be obtained from the previous frame in a temporal sequence of images, or from a depth camera with lower resolution or quality. Next, we determine a new normalized depth for every depth plane by analyzing the cumulative histogram. Steep sections of the cumulative histogram will result in a dense local distribution of planes; a flat section will result in a sparse distribution. The results, performed on controlled and on real images, demonstrate the effectiveness of the method over a uniform distribution and the possibility of using a lower number of depth planes, and thus a more performant processing, for the same quality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Goorts, P., Dumont, M., Rogmans, S., Bekaert, P.: An end-to-end system for free viewpoint video for smooth camera transitions. In: Proceedings of the Second International Conference on 3D Imaging (IC3D 2012), 3D Stereo Media (2012)
Goorts, P., Ancuti, C., Dumont, M., Bekaert, P.: Real-time video-based view interpolation of soccer events using depth-selective plane sweeping. In: Proceedings of the Eight International Conference on Computer Vision Theory and Applications (VISAPP 2013), INSTICC (2013)
Dumont, M., Rogmans, S., Maesen, S., Bekaert, P.: Optimized two-party video chat with restored eye contact using graphics hardware. In: Filipe, J., Obaidat, M.S. (eds.) ICETE 2008. CCIS, vol. 48, pp. 358–372. Springer, Heidelberg (2009)
Matusik, W., Buehler, C., Raskar, R., Gortler, S., McMillan, L.: Image-based visual hulls. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 369–374. ACM Press/Addison-Wesley Publishing Co. (2000)
Miller, G., Hilton, A., Starck, J.: Interactive free-viewpoint video. In: IEE European Conference on Visual Media Production, Citeseer, pp. 50–59 (2005)
Kutulakos, K., Seitz, S.: A theory of shape by space carving. Intl. J. Comput. Vis. 38, 199–218 (2000)
Seitz, S., Dyer, C.: Photorealistic scene reconstruction by voxel coloring. Intl. J. Comput. Vis. 35, 151–173 (1999)
Seitz, S., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 519–528. IEEE (2006)
Zitnick, C., Kang, S., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. ACM Trans. Graph. 23, 600–608 (2004). ACM
Yang, R., Pollefeys, M., Yang, H., Welch, G.: A unified approach to real-time, multi-resolution, multi-baseline 2d view synthesis and 3d depth estimation using commodity graphics hardware. Intl. J. Image Graph. 4, 627–651 (2004)
Rogmans, S., Dumont, M., Cuypers, T., Lafruit, G., Bekaert, P.: Complexity reduction of real-time depth scanning on graphics hardware. In: Proceedings of International Conference on Computer Vision Theory and Applications, Lisbon, Portugal, February 2009
Zhang, Z.: Microsoft kinect sensor and its effect. IEEE Multimedia 19, 4–10 (2012)
Gallup, D., Frahm, J.M., Mordohai, P., Yang, Q., Pollefeys, M.: Real-time plane-sweeping stereo with multiple sweeping directions. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR’07, pp. 1–8. IEEE (2007)
Goorts, P., Rogmans, S., Bekaert, P.: Raw camera image demosaicing using finite impulse response filtering on commodity gpu hardware using cuda. In: Proceedings of the Tenth International Conference on Signal Processing and Multimedia Applications (SIGMAP 2012), INSTICC (2012)
Yang, R., Welch, G., Bishop, G.: Real-time consensus-based scene reconstruction using commodity graphics hardware. Comput. Graph. Forum 22, 207–216 (2003). Wiley Online Library
Svoboda, T., Martinec, D., Pajdla, T.: A convenient multicamera self-calibration for virtual environments. Presence: Teleoperators Virtual Environ. 14, 407–422 (2005)
Green, S.: Image processing tricks in opengl. Presentation GDC 2005, p. 2 (2005)
Acknowledgements
Patrik Goorts would like to thank the IWT for its PhD specialization bursary.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
Plane sweeping of a soccer scene with a low number of depth planes (40) and a uniform plane distribution. Many artifacts and missing people can be perceived.
Plane sweeping of a soccer scene with a low number of depth planes (40) and an adaptive plane distribution. The quality is greatly increased in comparison with Fig. 10.
Plane sweeping of a soccer scene with a high number of depth planes (5000) and a uniform plane distribution. The quality is comparable with Fig. 11, which proves the effectiveness of the method.
(a) Kinect input image with one person, with the color image in the bottomright corner. (b) Cumulative histogram of the depth map. (c) New depth plane distribution for 10 planes. (d) Corresponding fraction \(\tau \) for a given plane number.
Comparison of the depth map with: (a) a uniform depth plane distribution (only 10 planes used), and (b) a non-uniform depth plane distribution using kinect (only 10 planes used).
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Goorts, P., Maesen, S., Dumont, M., Rogmans, S., Bekaert, P. (2014). Optimal Distribution of Computational Power in Free Viewpoint Interpolation by Depth Hypothesis Density Adaptation in Plane Sweeping. In: Obaidat, M., Filipe, J. (eds) E-Business and Telecommunications. ICETE 2013. Communications in Computer and Information Science, vol 456. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44788-8_20
Download citation
DOI: https://doi.org/10.1007/978-3-662-44788-8_20
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44787-1
Online ISBN: 978-3-662-44788-8
eBook Packages: Computer ScienceComputer Science (R0)