Dynamic Shape and Appearance Modeling Via Moving and Deforming Layers

  • Jeremy D. Jackson
  • Anthony Yezzi
  • Stefano Soatto
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3757)


We propose a model of the shape, motion and appearance of a sequence of images that captures occlusions, scene deformations, arbitrary viewpoint variations and changes in irradiance. This model is based on a collection of overlapping layers that can move and deform, each supporting an intensity function that can change over time. We discuss the generality and limitations of this model in relations to existing ones such as traditional optical flow or motion segmentation, layers, deformable templates and deformotion. We then illustrate how this model can be used for inference of shape, motion, deformation and appearance of the scene from a collection of images. The layering structure allows for automatic inpainting of partially occluded regions. We illustrate the model on synthetic and real sequences where existing schemes fail; we implement our gradient-based infinite-dimensional optimization using level set methods.


Active Contour Active Appearance Model Motion Segmentation Occlude Region Geodesic Active Contour 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alvarez, L., Weickert, J., Sanchez, J.: A scale-space approach to nonlocal optical flow calculations. In: Scale Space 1999, pp. 235–246 (1999)Google Scholar
  2. 2.
    Baker, S., Matthews, I., Schneider, J.: Image coding with active appearance models. Technical report, Carnegie Mellon University, The Robotics Institute (2003)Google Scholar
  3. 3.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Akeley, K. (ed.) Siggraph 2000, Computer Graphics Proceedings, pp. 417–424. ACM Press / ACM SIGGRAPH / Addison Wesley Longman (2000)Google Scholar
  4. 4.
    Blake, A., Isard, M.: Active contours. Springer, Heidelberg (1998)Google Scholar
  5. 5.
    Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int. J. of Computer Vision 22(1), 61–79 (1997)zbMATHCrossRefGoogle Scholar
  6. 6.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Proc. of the Eur. Conf. on Comp. Vis., pp. 484–496 (1998)Google Scholar
  7. 7.
    Cremers, D.: Multiphase levelset framework for variational motion segmentation. In: Intl. Conf. on Scale Space Theories in Computer Vision, June 2003, pp. 599–614 (2003)Google Scholar
  8. 8.
    Deriche, R., Kornprobst, P., Aubert, G.: Optical flow estimation while preserving its discontinuities: a variational approach. In: Proc. of ACCV, vol. 2, pp. 290–295 (1995)Google Scholar
  9. 9.
    Grenander, U.: General Pattern Theory. Oxford University Press, Oxford (1993)Google Scholar
  10. 10.
    Haussecker, H.W., Fleet, D.J.: Computing optical flow with physical models of brightness variation. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 661–673 (2001)CrossRefGoogle Scholar
  11. 11.
    Horn, B.K.P., Schunk, B.G.: Determining optical flow. Artificial Intell. 17, 185–203 (1981)CrossRefGoogle Scholar
  12. 12.
    Hsu, S., Anandan, P., Peleg, S.: Accurate computation of optical flow by using layered motion representations. In: Proc. IEEE Conf. on Comp. Vision and Pattern Recogn., pp. 1621–1626 (1992)Google Scholar
  13. 13.
    Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int. J. of Computer Vision 1(4), 321–331 (1987)CrossRefGoogle Scholar
  14. 14.
    Miller, M.I., Younes, L.: Group action, diffeomorphism and matching: a general framework. In: Proc. of SCTV (1999)Google Scholar
  15. 15.
    Osher, S., Sethian, J.: Fronts propagating with curvature-dependent speed: algorithms based on hamilton-jacobi equations. J. of Comp. Physics 79, 12–49 (1988)zbMATHMathSciNetCrossRefGoogle Scholar
  16. 16.
    Paragios, N., Deriche, R.: Geodesic active contours and level sets for the detection and tracking of moving objects. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(3), 266–280 (2000)CrossRefGoogle Scholar
  17. 17.
    Paragios, N., Rousson, M., Ramesh, V.: Non-rigid registration using distance functions. Comput. Vis. Image Underst. 89(2-3), 142–165 (2003)zbMATHCrossRefGoogle Scholar
  18. 18.
    Schnörr, C.: Computation of discontinuous optical flow by domain decomposition and shape optimization. Int. J. of Computer Vision 8(2), 153–165 (1992)CrossRefGoogle Scholar
  19. 19.
    Soatto, S., Yezzi, A.: Deformotion: deforming motion, shape average and the joint segmentation and registration of images. In: Proc. of the Eur. Conf. on Computer Vision (ECCV), vol. 3, pp. 32–47 (2002)Google Scholar
  20. 20.
    Wang, J., Adelson, E.: Representing moving images with layers. IEEE Trans. on Image Processing 3(5), 625–638 (1994)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Jeremy D. Jackson
    • 1
  • Anthony Yezzi
    • 1
  • Stefano Soatto
    • 2
  1. 1.School of Electrical and Computer EngineeringGeorgia Institute of TechnologyAtlantaUSA
  2. 2.Computer Science DepartmentUCLALos AngelesUSA

Personalised recommendations