Bayesian Estimation of Layers from Multiple Images

  • Y. Wexler
  • A. Fitzgibbon
  • A. Zisserman
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2352)


When estimating foreground and background layers (or equivalently an alpha matte), it is often the case that pixel measurements contain mixed colours which are a combination of foreground and background. Object boundaries, especially at thin sub-pixel structures like hair, pose a serious problem.

In this paper we present a multiple view algorithm for computing the alpha matte. Using a Bayesian framework, we model each pixel as a combined sample from the foreground and background and compute a MAP estimate to factor the two. The novelties in this work include the incorporation of three different types of priors for enhancing the results in problematic scenes. The priors used are inequality constraints on colour and alpha values, spatial continuity, and the probability distribution of alpha values.

The combination of these priors result in accurate and visually satisfying estimates. We demonstrate the method on real image sequences with varying degrees of geometric and photometric complexity. The output enables virtual objects to be added between the foreground and background layers, and we give examples of this augmentation to the original sequences.


Ground Truth Bayesian Estimation Multiple Image Virtual Object Foreground Object 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    C. Barron. Matte painting in the digital age. In Proc. SIGGRAPH (sketches), 1998.
  2. 2.
    P. Beardsley, P. Torr, and A. Zisserman. 3D model acquisition from extended image sequences. In Proc. ECCV, LNCS 1064/1065, pages 683–695. Springer-Verlag, 1996.Google Scholar
  3. 3.
    M. J. Black and P. Anandan. A framework for the robust estimation of optical flow. In Proc. ICCV, pages 231–236, 1993.Google Scholar
  4. 4.
    P. T. Boggs and J. W. Tolle. Sequential quadratic programming. In Acta Numerica, pages 1–51. 1995.Google Scholar
  5. 5.
    Yung-Yu Chuang, Brian Curless, David Salesin, and Richard Szeliski. A bayesian approach to digital matting. In CVPR 2001, 2000.Google Scholar
  6. 6.
    P. J. Huber. Robust Statistics. John Willey and Sons, 1981.Google Scholar
  7. 7.
    M. Irani, B. Rousso, and S. Peleg. Detecting and tracking multiple moving objects using temporal integration. In G. Sandini editor, Proc. ECCV, pages 282–287. Springer-Verlag, 1992.Google Scholar
  8. 8.
    H. H. Nagel. Extending the’ oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow. In O. D. Faugeras, editor, Proc. ECCV, pages 139–148. Springer-Verlag, 1990.Google Scholar
  9. 9.
    S. Peleg and J. Herman. Panoramic mosaics by manifold projection. In Proc. CVPR, 1997.Google Scholar
  10. 10.
    P. Perez, A. Blake, and M. Gangnet. Jetstream: Probabilistic contour extraction with particles. In International Conference on Computer Vision, pages 424–531, 2001.Google Scholar
  11. 11.
    P. Perona and J. Malik. Scale-space and edge detection using anisotropic diffusion. IEEE PAMI, 12(7):629–639, Jul 1990.Google Scholar
  12. 12.
    M. Shizawa and K. Mase. A unified computational theory for motion transparency and motion boundaries based on eigenenergy analysis. In CVPR91, pages 289–295, 1991.Google Scholar
  13. 13.
    A. R. Smith and J. F. Blinn. Blue screen matting. In Proc. SIGGRAPH, pages 259–268, 1996.Google Scholar
  14. 14.
    R. Szeliski, S. Avidan, and P. Anandan. Layer extraction from multiple images containing reflections and transparency. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’2000), volume 1, pages 246–253, 2000.Google Scholar
  15. 15.
    R. Szeliski and P. Golland. Stereo matching with transparency and matting. In Sixth International Conference on Computer Vision (ICCV’98), pages 517–524, 1998.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Y. Wexler
    • 1
  • A. Fitzgibbon
    • 1
  • A. Zisserman
    • 1
  1. 1.Robotics Research Group Department of Engineering ScienceUniversity of OxfordOxford

Personalised recommendations