Skip to main content
SpringerLink
Log in

Bayesian Approaches to Motion-Based Image and Video Segmentation

  • Conference paper
Book cover Complex Motion (IWCM 2004)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 3417))

Included in the following conference series:

Abstract

We present a variational approach for segmenting the image plane into regions of piecewise parametric motion given two or more frames from an image sequence. Our model is based on a conditional probability for the spatio-temporal image gradient, given a particular velocity model, and on a geometric prior on the estimated motion field favoring motion boundaries of minimal length.

We cast the problem of motion segmentation as one of Bayesian inference, we derive a cost functional which depends on parametric motion models for each of a set of domains and on the boundary separating them. The resulting functional can be interpreted as an extension of the Mumford-Shah functional from intensity segmentation to motion segmentation. In contrast to most alternative approaches, the problems of segmentation and motion estimation are jointly solved by continuous minimization of a single functional. Minimization results in an eigenvalue problem for the motion parameters and in a gradient descent evolution for the motion boundary. The evolution of the motion boundaries is implemented by a multiphase level set formulation which allows for the segmentation of an arbitrary number of multiply connected moving objects.

We further extend this approach to the segmentation of space-time volumes of coherent motion from video sequences. To this end, motion boundaries are represented by a set of surfaces in space-time. An implementation by a higher-dimensional multiphase level set model allows the evolving surfaces to undergo topological changes. In contrast to an iterative segmentation of consecutive frame pairs, a constraint on the area of these surfaces leads to an additional temporal regularization of the computed motion boundaries.

Numerical results demonstrate the capacity of our approach to segment objects based exclusively on their relative motion.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bigün, J., Granlund, G.H., Wiklund, J.: Multidimensional orientation estimation with applications to texture analysis and optical flow. IEEE PAMI 13(8), 775–790 (1991)

    Google Scholar 

  2. Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise–smooth flow fields. Comp. Vis. Graph. Image Proc.: IU 63(1), 75–104 (1996)

    Google Scholar 

  3. Bouthemy, P., Francois, E.: Motion segmentation and qualitative dynamic scene analysis from an image sequence. Int. J. of Comp. Vis. 10(2), 157–182 (1993)

    Google Scholar 

  4. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)

    Google Scholar 

  5. Caselles, V., Coll, B.: Snakes in movement. SIAM J. Numer. Anal. 33, 2445–2456 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  6. Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. In: Proc. IEEE Intl. Conf. on Comp. Vis., Boston, USA, pp. 694–699. IEEE, Los Alamitos (1995)

    Chapter  Google Scholar 

  7. Chan, T., Vese, L.: Active contours without edges. IEEE Trans. Image Processing 10(2), 266–277 (2001)

    Article  MATH  Google Scholar 

  8. Cremers, D.: A multiphase level set framework for variational motion segmentation. In: Griffin, L.D, Lillholm, M. (eds.) Scale Space Methods in Computer Vision. LNCS, vol. 2695, pp. 599–614. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  9. Cremers, D.: A variational framework for image segmentation combining motion estimation and shape regularization. In: Dyer, C., Perona, P. (eds.) IEEE Conf. on Comp. Vis. and Patt. Recog, vol. 1, pp. 53–58. IEEE Computer Society Press, Los Alamitos (June 2003)

    Google Scholar 

  10. Cremers, D., Schnörr, C.: Motion Competition: Variational integration of motion segmentation and shape regularization. In: Van Gool, L. (ed.) DAGM 2002. LNCS, vol. 2449, pp. 472–480. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  11. Cremers, D., Soatto, S.: Variational space-time motion segmentation. In: Triggs, B., Zisserman, A. (eds.) IEEE Int. Conf. on Computer Vision, Nice, pp. 886–892. IEEE, Los Alamitos (Oct. 2003)

    Chapter  Google Scholar 

  12. Cremers, D., Soatto, S.: Motion Competition: A variational framework for piecewise parametric motion segmentation. Int. J. of Comp. Vis. (to appear, 2004)

    Google Scholar 

  13. Cremers, D., Yuille, A.L.: A generative model based approach to motion segmentation. In: Michaelis, B., Krell, G. (eds.) Pattern Recognition. LNCS, vol. 2781, pp. 313–320. Springer, Heidelberg (2003)

    Google Scholar 

  14. Farnebäck, G.: Very high accuracy velocity estimation using orientation tensors, parametric motion, and segmentation of the motion field. In: ICCV, vol. 1, pp. 171–177 (2001)

    Google Scholar 

  15. Heitz, F., Bouthemy, P.: Multimodal estimation of discontinuous optical flow using markov random fields. IEEE PAMI 15(12), 1217–1232 (1993)

    Google Scholar 

  16. Horn, B.K.P., Schunck, B.G.: Determining optical flow. A.I. 17, 185–203 (1981)

    Article  Google Scholar 

  17. Jehan-Besson, S., Barlaud, M., Aubert, G.: DREAM2S: Deformable regions driven by an eulerian accurate minimization method for image and video segmentation. Int. J. of Comp. Vis. 53(1), 45–70 (2003)

    Google Scholar 

  18. Jepson, A., Black, M.J.: Mixture models for optic flow computation. In: Proc. IEEE Conf. on Comp. Vision Patt. Recog., New York, pp. 760–761. IEEE Computer Society Press, Los Alamitos (1993)

    Google Scholar 

  19. Kichenassamy, S., Kumar, A., Olver, P.J., Tannenbaum, A., Yezzi, A.J.: Gradient flows and geometric active contour models. In: Proc. IEEE Intl. Conf. on Comp. Vis., Boston, USA, pp. 810–815. IEEE, Los Alamitos (1995)

    Chapter  Google Scholar 

  20. Konrad, J., Dubois, E.: Bayesian estimation of motion vector fields. IEEE PAMI 14(9), 910–927 (1992)

    Google Scholar 

  21. Kornprobst, P., Deriche, R., Aubert, G.: Image sequence analysis via partial differential equations. J. Math. Im. Vis. 11(1), 5–26 (1999)

    Article  MathSciNet  Google Scholar 

  22. Memin, E., Perez, P.: Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Trans. on Im. Proc. 7(5), 703–719 (1998)

    Article  Google Scholar 

  23. Morel, J.-M., Solimini, S.: Variational Methods in Image Segmentation. Birkhäuser, Boston (1995)

    Google Scholar 

  24. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math. 42, 577–685 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  25. Nagel, H.H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE PAMI 8(5), 565–593 (1986)

    Google Scholar 

  26. Nestares, O., Fleet, D.J., Heeger, D.J.: Likelihood functions and confidence bounds for total-least-squares problems. In: Proc. Conf. Computer Vis. and Pattern Recog., vol. 1, Hilton Head Island, SC, June, pp. 760–767 (2000)

    Google Scholar 

  27. Odobez, J.-M., Bouthemy, P.: Direct incremental model-based image motion segmentation for video analysis. Signal Proc. 66, 143–155 (1998)

    Article  MATH  Google Scholar 

  28. Osher, S.J., Sethian, J.A.: Fronts propagation with curvature dependent speed: Algorithms based on Hamilton–Jacobi formulations. J. of Comp. Phys. 79, 12–49 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  29. Paragios, N., Deriche, R.: Geodesic active contours and level sets for the detection and tracking of moving objects. IEEE PAMI 22(3), 266–280 (2000)

    Google Scholar 

  30. Schnörr, C.: Computation of discontinuous optical flow by domain decomposition and shape optimization. Int. J. of Comp. Vis. 8(2), 153–165 (1992)

    Google Scholar 

  31. Shi, J., Malik, J.: Motion segmentation and tracking using normalized cuts. In: Intl. Conf. on Comp. Vision, Bombay, India (1998)

    Google Scholar 

  32. Sussman, M., Smereka, P., Osher, S.J.: A level set approach for computing solutions to incompressible twophase flow. J. of Comp. Phys. 94, 146–159 (1994)

    Article  Google Scholar 

  33. Torr, P.H.S., Szeliski, R., Anandan, P.: An integrated bayesian approach to layer extraction from image sequences. IEEE PAMI 23(3), 297–303 (2002)

    Google Scholar 

  34. Vidal, R., Ma, Y., Sastry, S.: Generalized principal component analysis (gpca): an analytic solution to segmentation of mixtures of subspaces. In: Proc. IEEE Conf. on Comp. Vision Patt. Recog., vol. 1, Madison, pp. 621–628. IEEE Computer Society Press, Los Alamitos (2003)

    Google Scholar 

  35. Wang, J.Y.A., Adelson, E.H.: Representating moving images with layers. IEEE Trans. on Image Processing 3(5), 625–638 (1994)

    Article  Google Scholar 

  36. Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in PDE–based computation of image motion. Int. J. of Comp. Vis. 45(3), 245–264 (2001)

    MATH  Google Scholar 

  37. Weiss, Y.: Smoothness in layers: Motion segmentation using nonparametric mixture estimation. In: Proc. IEEE Conf. on Comp. Vision Patt. Recog., Puerto Rico, pp. 520–527. IEEE Computer Society Press, Los Alamitos (1997)

    Google Scholar 

  38. Weiss, Y., Fleet, D.J.: Velocity likelihoods in biological and machine vision. In: Lewicki, M.S., Rao, R.P.N., Olshausen, B.A. (eds.) Probabilistic Models of the Brain: Perception and Neural Function, pp. 81–100. MIT Press, Cambridge (2001)

    Google Scholar 

  39. Zhu, S.C., Yuille, A.: Region competition: Unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE PAMI 18(9), 884–900 (1996)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of California, Los Angeles, USA

    Daniel Cremers

Authors
  1. Daniel Cremers
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Bernd Jähne Rudolf Mester Erhardt Barth Hanno Scharr

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer Berlin Heidelberg

About this paper

Cite this paper

Cremers, D. (2007). Bayesian Approaches to Motion-Based Image and Video Segmentation. In: Jähne, B., Mester, R., Barth, E., Scharr, H. (eds) Complex Motion. IWCM 2004. Lecture Notes in Computer Science, vol 3417. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69866-1_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-69866-1_9

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-69864-7

  • Online ISBN: 978-3-540-69866-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation