Abstract
Real-time stereo matching has many important applications in areas such as robotic navigation and immersive teleconferencing. When processing stereo sequences most existing real-time stereo algorithms calculate disparity maps for different frames independently without considering temporal consistency between adjacent frames. While it is known that temporal consistency information can help to produce better results, there is no efficient way to enforce temporal consistency in real-time applications.
In this paper the temporal correspondences between disparity maps of adjacent frames are modeled using a new concept called disparity flow. A disparity flow map for a given view depicts the 3D motion in the scene that is observed from this view. An algorithm is developed to compute both disparity maps and disparity flow maps in an integrated process. The disparity flow map generated for the current frame is used to predict the disparity map for the next frame and hence, the temporal consistency between the two frames is enforced. All computations are performed in the image space of the given view, leading to an efficient implementation. In addition, most calculations are executed on programmable graphics hardware which further accelerates the processing speed. The current implementation can achieve 89 million disparity estimations per second on an ATI Radeon X800 graphic card. Experimental results on two stereo sequences demonstrate the effectiveness of the algorithm.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Davis, J., Nehab, D., Ramamoothi, R., Rusinkiewicz, S.: Spacetime stereo: a unifying framework for depth from triangulation. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (2005)
Dhond, U.R., Aggarwal, J.K.: Structure from stereo - A review. IEEE Transactions on Systems, Man and Cybernetics 19, 1489–1510 (1989)
Forstmann, S., Ohya, J., Kanou, Y., Schmitt, A., Thuering, S.: Real-time stereo by using dynamic programming. In: Proc. CVPR Workshop on Real-time 3D Sensors and Their Use, Washington, DC, USA, pp. 29–36 (2004)
Gong, M., Yang, R.: Image-gradient-guided real-time stereo on graphics hardware. In: Proc. International Conference on 3-D Digital Imaging and Modeling, Ottawa, ON, Canada, pp. 548–555 (2005)
Gong, M., Yang, Y.-H.: Near real-time reliable stereo matching using programmable graphics hardware. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, pp. 924–931 (2005)
Hirschmuller, H., Innocent, P.R., Garibaldi, J.: Real-time correlation-based stereo vision with reduced border errors. International Journal of Computer Vision 47 (2002)
Hong, L., Chen, G.: Segment-based stereo matching using graph cuts. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, pp. 74–81 (2004)
Kanade, T., Yoshida, A., Oda, K., Kano, H., Tanaka, M.: A stereo engine for videorate dense depth mapping and its new applications. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 196–202 (1996)
Leung, C., Appleton, B., Lovell, B.C., Sun, C.: An energy minimisation approach to stereo-temporal dense reconstruction. In: Proc. International Conference on Pattern Recognition, Cambridge, UK, pp. 72–75 (2004)
Okutomi, M., Kanade, T.: A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 15, 353–363 (1993)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 47, 7–42 (2002)
Sun, J., Li, Y., Kang, S.B., Shum, H.-Y.: Symmetric stereo matching for occlusion handling. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, SanDiego, CA, USA, pp. 399–406 (2005)
Tao, H., Sawhney, H.S., Kumar, R.: Dynamic depth recovery from multiple synchronized video streams. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA (2001)
Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. In: Proc. International Conference on Computer Vision (1999)
Woetzel, J., Koch, R.: Real-time multi-stereo depth estimation on GPU with approximative discontinuity handling. In: Proc. European Conference on Visual Media Production, London, United Kingdom (2004)
Yang, R., Pollefeys, M.: Multi-resolution real-time stereo on commodity graphics hardware. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Madison,WI, USA, pp. 211–220 (2003)
Yang, R., Pollefeys, M., Li, S.: Improved real-time stereo on commodity graphics hardware. In: Proc. CVPR Workshop on Real-time 3D Sensors and Their Use, Washington, DC, USA (2004)
Zhang, L., Curless, B., Seitz, S.M.: Spacetime stereo: shape recovery for dynamic scenes. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Madison, WI, USA, pp. 367–374 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gong, M. (2006). Enforcing Temporal Consistency in Real-Time Stereo Estimation. In: Leonardis, A., Bischof, H., Pinz, A. (eds) Computer Vision – ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3953. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11744078_44
Download citation
DOI: https://doi.org/10.1007/11744078_44
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33836-9
Online ISBN: 978-3-540-33837-6
eBook Packages: Computer ScienceComputer Science (R0)