Abstract
We address the problem of detecting occlusion boundaries from motion sequences, which is important for motion segmentation, estimating depth order, and related tasks. Previous work by Stein and Hebert has addressed this problem and obtained good results on a benchmarked dataset using two-dimensional image cues, motion estimation, and a global boundary model [1]. In this paper we describe a method for detecting occlusion boundaries which uses depth cues and local segmentation cues. More specifically, we show that crude scaled estimates of depth, which we call pseudo-depth, can be extracted from motion sequences containing a small number of image frames using standard SVD factorization methods followed by weak smoothing using a Markov Random Field defined over super-pixels. We then train a classifier for occlusion boundaries using pseudo-depth and local static boundary cues (adding motion cues only gives slightly better results). We evaluate performance on Stein and Hebert’s dataset and obtain results of similar average quality which are better in the low recall/high precision range. Note that our cues and methods are different from [1] – in particular we did not use their sophisticated global boundary model – and so we conjecture that a unified approach would yield even better results.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Stein, A., Hebert, M.: Occlusion boundaries from motion: Low-level detection and mid-level reasoning. International Journal of Computer Vision 82, 325–357 (2009)
Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. In: ICCV (2007)
Anandan, P.: A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision 2, 283–310 (1989)
Roth, S., Black, M.J.: On the spatial statistics of optical flow. International Journal of Computer Vision 74, 1 (2007)
Wedel, A., Cremers, D., Pock, T., Bischof, H.: Structure- and motion-adaptive regularization for high accuracy optic flow. In: ICCV (2009)
Geiger, D., Ladendorf, B., Yuille, A.: Occlusions and binocular stereo. International Journal of Computer Vision 14, 211–226 (1995)
Belhumeur, P., Mumford, D.: A bayesian treatment of the stereo correspondence problem using half-occluded regions, pp. 506–512 (1992)
Gamble, E., Geiger, D., Poggio, T., Weinshall, D.: Integration of vision modules and labeling of surface discontinuities. IEEE Transactions on Systems, Man and Cybernetics 19, 1576–1581 (1989)
Kontsevich, L.L., Kontsevich, M.L., Shen, A.K.: Two algorithms for reconstructing shapes. Optoelectronics, Instrumentation and Data Processing 5, 75–81 (1987)
Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. International Journal of Computer Vision 9, 2 (1992)
Ren, X., Malik, J.: Learning a classification model for segmentation. In: ICCV (2003)
Saxena, A., Sun, M., Ng, A.Y.: Make3d: Learning 3d scene structure from a single still image. IEEE Trans. Pattern Anal. Mach. Intell. 31, 824–840 (2009)
Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Anal. Mach. Intell. 26, 530–549 (2004)
Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)
Black, M., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. CVIU 63, 1 (1996)
Fleet, D.J., Black, M.J., Nestares, O.: Bayesian inference of visual motion boundaries. In: Exploring artificial intelligence in the new millenniu, pp. 139–173 (2003)
Zitnick, C.L., Jojic, N., Kang, S.B.: Consistent segmentation for optical flow estimation. In: ICCV (2005)
Barbu, A., Yuille, A.: Motion estimation by swendsen-wang cuts. In: CVPR (2004)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)
Agarwal, S., Snavely, N., Simon, I., Seitz, S.M., Szeliski, R.: Building rome in a day. In: ICCV (2009)
Xiong, Y., Shafer, S.A.: Dense structure from a dense optical flow sequence. Comput. Vis. Image Underst. 69, 222–245 (1998)
Ernst, F., Wilinski, P., van Overveld, C.W.A.M.: Dense structure-from-motion: An approach based on segment matching. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 217–231. Springer, Heidelberg (2002)
Calway, A.: Recursive estimation of 3d motion and surface structure from local affine flow parameters. IEEE Trans. Pattern Anal. Mach. Intell. 27, 562–574 (2005)
Russell, B.C., Torralba, A.: Building a database of 3d scenes from user annotations. In: CVPR (2009)
Hoiem, D., Stein, A., Efros, A., Hebert, M.: Recovering occlusion boundaries from a single image. In: ICCV (2007)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)
Konishi, S., Yuille, A., Coughlan, J., Zhu, S.C.: Statistical edge detection: learning and evaluating edge cues. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 57–74 (2003)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)
Costeira, J., Kanade, T.: A multibody factorization method for independently moving-objects 29, 159–179 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
He, X., Yuille, A. (2010). Occlusion Boundary Detection Using Pseudo-depth. In: Daniilidis, K., Maragos, P., Paragios, N. (eds) Computer Vision – ECCV 2010. ECCV 2010. Lecture Notes in Computer Science, vol 6314. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15561-1_39
Download citation
DOI: https://doi.org/10.1007/978-3-642-15561-1_39
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15560-4
Online ISBN: 978-3-642-15561-1
eBook Packages: Computer ScienceComputer Science (R0)