Abstract
New applications in fields such as augmented or virtualized reality have created a demand for dense, accurate real-time stereo reconstruction. Our goal is to reconstruct a user and her office environment for networked tele-immersion, which requires accurate depth values in a relatively large workspace. In order to cope with the combinatorics of stereo correspondence we can exploit the temporal coherence of image sequences by using coarse optical flow estimates to bound disparity search ranges at the next iteration. We use a simple flood fill segmentation method to cluster similar disparity values into overlapping windows and predict their motion over time using a single optical flow calculation per window. We assume that a contiguous region of disparity represents a single smooth surface which allows us to restrict our search to a narrow disparity range. The values in the range may vary over time as objects move nearer or farther away in Z, but we can limit the number of disparities to a feasible search size per window. Further, the disparity search and optical flow calculation are independent for each window, and allow natural distribution over a multi-processor architecture.
We have examined the relative complexity of stereo correspondence on full images versus our proposed window system and found that, depending on the number of frames in time used to estimate optical flow, the window-based system requires about half the time of standard correlation stereo. Experimental comparison to full image correspondence search shows our window-based reconstructions compare favourably to those generated by the full algorithm, even after several frames of propagation via estimated optical flow. The result is a system twice as fast as conventional dense correspondence without significant degradation of extracted depth values.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Nicholas Ayache. Artificial Vision for Mobile Robots: Stereo Vision and Multisensory Perception. The MIT Press, Cambridge, MA, 1991.
Poornima Balasubramanyam. The p-field: A computational model for binocular motion processing. In 1991 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’91), pages 115–120, 1991.
P.A. Beardsley, J.M. Brady, and D.W. Murray. Prediction of stereo disparity using optical flow. In British Machine Vision Conf., pages 259–264, 1990.
S. S. Beauchemin and J. L. Barron. The computation of optical flow. ACM Computing Surveys, 27(3):433–467, 1995.
Peter Burt, Lambert Wixson, and Garbis Salgian. Electronically directed “focal” stereo. In 5th Intl. Conf. on Computer Vision (ICCV95), pages 94–101, Cambridge, Mass., June 1995.
U. Dhond and J. Aggarwal. Structure from stereo-a review. IEEE Trans. on Systems, Man and Cybernetics (SMC), 19(6):1489–1510, Nov/Dec 1989.
Michal Irani, Benny Rousso, and Shmuel Peleg. Computing occluding transparent motions. Intl. Journal of Computer Vision, 12(1):5–16, January 1994.
Takeo Kanade, Atsuchi Yoshida, Kazuo Oda, Hiroshi Kano, and Masaya Tanaka. A stereo machine for video-rate dense depth mapping and its new applications. In 1996 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’96), pages 196–202, San Francisco, CA, June 1996.
Kurt Konolige. Small vision systems: Hardware and implmentation. In Eighth Intl. Symposium on Robotics Research (Robotics Research 8), pages 203–212, Hayama, Japan, Oct. 1997.
Stephane Laveau and Olivier Faugeras. 3-d scene representation as a collection of images and fundamental matrices. Technical Report 2205, INRIA, February 1994.
B. D. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In 7th Intl. Joint Conf. on Artificial Intelligence (IJCAI’81), pages 674–679, Vancouver, BC, 1981.
R. Mandelbaum, M. Hansen, P. Burt, and S. Baten. Vision for autonomous mobility: Image processing on the vfe-200. In IEEE Intl. Symposium on Intelligent Control (ISIC), Intl. Symposium on Computational Intelligence in Robotics and Automation (CIRA), and Intelligent Systems and Semiotics (ISAS), Gaithersburg, MD, September 1998.
Francois Meyer and Patrick Bouthemy. Region-based tracking in an image sequence. In G. Sandini, editor, Computer Vision-ECCV’92: Second European Conf. on Computer Vision, volume 588 of Lecture Notes in Computer Science, pages 476–484, Santa Margherita Ligure, Italy, May 1992. Springer-Verlag.
Hans-Hellmut Nagel. Dynamic stereo vision in a robot feedback loop based on the evaluation of multiple interconnected displacement vector fields. In Robotics Research: The Third Intl. Symposium, pages 49–55, Gouvieux, France, Oct. 1985.
P. J. Narayanan and Takeo Kanade. Virtual worlds using computer vision. In IEEE/ATR Workshop on Comp. Vis. for VR based Human Comm. (CVVRHC’98), pages 2–13, Bombay, India, January 1998.
William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery. Numerical Recipes in C: The Art of Scientific Computing Cambridge University Press, Cambridge, UK, second edition, 1992.
David F. Rogers. Procedural Elements for Computer Graphics. WCB/McGraw-Hill, Boston, MA, second edition, 1998.
Steven M. Seitz and Charles R. Dyer. Toward image-based scene representation using view morphing. In 13th IEEE Conf. on Computer Vision and Pattern Recognition, Vienna, Austria, 1996.
Amnon Shashua. Algebraic functions for recognition. EEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 17(8):779–789, 1995.
M. Tistarelli, E. Grosso, and G. Sandini. Dynamic stereo in visual navigation. In 1991 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’91), pages 186–193, 1991.
Emanuele Trucco and Alessandro Verri. Introductory Techniques for 3-D Computer Vision. Prentice Hall Inc., Upper Saddle River, NJ, 1998.
Vladimir Tucakov and David. G. Lowe. Temporally coherent stereo: Improving performance through knowledge of motion. In 1997 IEEE Intl. Conf. on Robotics and Automation (ICRA’97), pages 1999–2006, Albuquerque, NM, April 1997.
Allen M. Waxman and James H. Duncan. Binocular image flows: Steps toward stereo-motion fusion. IEEE Trans. on Pattern Analysis and Machine Intelligence, PAMI-8(6):715–729, November 1986.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mulligan, J., Daniilidis, K. (2000). Predicting Disparity Windows for Real-Time Stereo. In: Computer Vision - ECCV 2000. ECCV 2000. Lecture Notes in Computer Science, vol 1842. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45054-8_15
Download citation
DOI: https://doi.org/10.1007/3-540-45054-8_15
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67685-0
Online ISBN: 978-3-540-45054-2
eBook Packages: Springer Book Archive