Image sequence restoration: A PDE based coupled method for image restoration and motion segmentation

  • Pierre Kornprobst
  • Rachid Deriche
  • Gilles Aubert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1407)


This article deals with the problem of restoring and segmenting noisy image sequences with a static background. Usually, motion segmentation and image restoration are tackled separately in image sequence restoration. Moreover, segmentation is often noise sensitive. In this article, the motion segmentation and the image restoration parts are performed in a coupled way, allowing the motion segmentation part to positively influence the restoration part and vice-versa. This is the key of our approach that allows to deal simultaneously with the problem of restoration and motion segmentation. To this end, we propose a theoretically justified optimization problem that permits to take into account both requirements. A suitable numerical scheme based on half quadratic minimization is then proposed and its stability demonstrated. Experimental results obtained on noisy synthetic data and real images will illustrate the capabilities of this original and promising approach.


Image Sequence Image Restoration Anisotropic Diffusion Restoration Part Motion Segmentation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    T. Aach and A. Kaup. Bayesian algorithms for adaptive change detection in image sequences using markov random fields. Signal Processing: Image Communication, 7:147–160, 1995.CrossRefGoogle Scholar
  2. 2.
    L. Alvarez and L. Mazorra. Signal and image restoration using shock filters and anisotropic diffusion. SIAM Journal of numerical analysis, 31(2):590–605, Apr. 1994.zbMATHMathSciNetCrossRefGoogle Scholar
  3. 3.
    G. Aubert, M. Barlaud, L. Blanc-Feraud, and P. Charbonnier. Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Imag. Process., 5(12), Feb. 1997.Google Scholar
  4. 4.
    G. Aubert and L. Vese. A variational method in image recovery. SIAM J. Numer. Anal., 34(5):1948–1979, Oct. 1997.zbMATHMathSciNetCrossRefGoogle Scholar
  5. 5.
    J. Boyce. Noise reduction of image sequences using adaptative motion compensated frame averaging. In IEEE ICASSP, volume 3, pages 461–464, 1992.Google Scholar
  6. 6.
    J. Brailean and A. Katsaggelos. Simultaneous recursive displacement estimation and restoration of noisy-blurred image sequences. IEEE Transactions on Image Processing, 4(9):1236–1251, Sept. 1995.CrossRefGoogle Scholar
  7. 7.
    O. Buisson, B. Besserer, S. Boukir, and F. Helt. Deterioration detection for digital film restoration. In Computer Vision and Pattern Recognition, pages 78–84, Puerto Rico, June 1997.Google Scholar
  8. 8.
    L. D. Cohen. Auxiliary variables and two-step iterative algorithms in computer vision problems. ICCV, 1995.Google Scholar
  9. 9.
    R. Deriche and O. Faugeras. Les EDP en traitement des images et vision par ordinateur. Technical report, INRIA, Nov. 1995. A more complete version of this Research Report has appeared in the French Revue “Traitement du Signal”. Volume 13 — No 6 — Special 1996.Google Scholar
  10. 10.
    N. Diehl. Object-oriented motion estimation and segmentation in image sequences. IEEE Transactions on Image Processing, 3:1901–1904, Feb. 1990.Google Scholar
  11. 11.
    E. Dubois and S. Sabri. Noise reduction in image sequences using motion-compensated temporal filtering. IEEE Transactions on Communications, 32(7):826–831, July 1984.CrossRefGoogle Scholar
  12. 12.
    L. C. Evans and R. F. Gariepy. Measure Theory and Fine Properties of Functions. CRC, 1992.Google Scholar
  13. 13.
    D. Geman and G. Reynolds. Constrained restoration and the recovery of discontinuities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(3):367–383, 1993.CrossRefGoogle Scholar
  14. 14.
    S. Geman, D. E. McClure, and D. Geman. A nonlinear filter for film restoration and other problems in image processing. CVGIP: Graphical Models and Image Processing, 54(4):281–289, July 1992.CrossRefGoogle Scholar
  15. 15.
    K. Karmann, A. Brandt, and R. Gerl. Moving object segmentation based on adaptive reference images. Signal Processing: Theories and Applications, V:951–954, 1990.Google Scholar
  16. 16.
    A. Kokaram. Reconstruction of severely degraded image sequences. In International Conference on Image Applications and Processing, Florence, Italy, 1997.Google Scholar
  17. 17.
    A. C. Kokaram and S. Godsill. A system for reconstruction of missing data in image sequences using sampled 3d ar models and mrf motion priors. In B. Buxton, editor, Proceedings of the 4th European Conference on Computer Vision, pages 613–624, Cambridge, UK, Apr. 1996.Google Scholar
  18. 18.
    P. Kornprobst, G. Aubert, and R. Deriche. A ivariational method for image sequences interpretation (submitted). 1997.Google Scholar
  19. 19.
    P. Kornprobst, R. Deriche, and G. Aubert. Image coupling, restoration and enhancement via PDE's. In International Conference on Image Processing, volume II of III, pages 458–461, Santa-Barbara, California, Oct. 1997.Google Scholar
  20. 20.
    P. Kornprobst, R. Deriche, and G. Aubert. Image Sequence Restoration: A PDE Based Coupled Method for Image Restoration and Motion Segmentation. Technical Report 3308, INRIA, Nov. 1997.Google Scholar
  21. 21.
    P. Kornprobst, R. Deriche, and G. Aubert. Nonlinear operators in image restoration. In Proceedings of the International Conference on Computer Vision and Pattern Recognition, pages 325–331, Puerto-Rico, June 1997. IEEE.Google Scholar
  22. 22.
    S. Liou and R. Jain. Motion detection in spatio-temporal space. Computer Vision, Graphics and Image Understanding, (45):227–250, 1989.Google Scholar
  23. 23.
    R. Malladi and J. Sethian. Image processing: Flows under min/max curvature and mean curvature. Graphical Models and Image Processing, 58(2):127–141, Mar. 1996.MathSciNetCrossRefGoogle Scholar
  24. 24.
    R. Morris. Image Sequence Restoration using Gibbs Distributions. PhD thesis, Cambridge University, England, 1995.Google Scholar
  25. 25.
    N. Paragios and R. Deriche. A PDE-based Level Set Approach for Detection and Tracking of Moving Objects. In Proceedings of the 6th International Conference on Computer Vision, Bombay, India, Jan. 1998. IEEE Computer Society Press.Google Scholar
  26. 26.
    N. Paragios and G. Tziritas. Detection and localization of moving objects in image sequences. FORT-Hellas Technical Report, Accepted for publication in Signal Processing: Image Communication, Oct. 1996.Google Scholar
  27. 27.
    P. Perona and J. Malik. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7):629–639, July 1990.CrossRefGoogle Scholar
  28. 28.
    M. Proesmans, E. Pauwels, and L. V. Gool. Coupled Geometry-Driven Diffusion Equations for Low-Level Vision, pages 191–228. Computational imaging and vision. Kluwer Academic Publishers, 1994.Google Scholar
  29. 29.
    L. Rudin and S. Osher. Total variation based image restoration with free local constraints. In International Conference on Image Processing, volume I, pages 31–35, Nov. 1994.Google Scholar
  30. 30.
    G. Sapiro, A. Tannenbaum, Y. You, and M. Kaveh. Experiments on geometric image enhancement. In International Conference on Image Processing, 1994.Google Scholar
  31. 31.
    C. Schnörr. Unique reconstruction of piecewise-smooth images by minimizing strictly convex nonquadratic functionals. Journal of Mathematical Imaging and Vision, 4:189–198, 1994.MathSciNetCrossRefGoogle Scholar
  32. 32.
    R. Stevenson, B. Schmitz, and E. Delp. Discontinuity preserving regularization of inverse visual problems. IEEE Transactions on Systems, Man, and Cybernetics, 24(3):455–469, Mar. 1994.CrossRefGoogle Scholar
  33. 33.
    L. Vese. Problèmes variationnels et EDP pour l'analyse d'images et l'évolution de courbes. PhD thesis, Université de Nice Sophia-Antipolis, Nov. 1996.Google Scholar
  34. 34.
    J. Weickert. Anisotropic Diffusion in Image Processing. PhD thesis, University of Kaiserslautern, Germany, Laboratory of Technomathematics, Jan. 1996.Google Scholar
  35. 35.
    O. Wenstop. Motion detection from image information. Proceedings in Scandianvian Conference on Image Analysis, pages 381–386, 1983.Google Scholar
  36. 36.
    Y. You, M. Kaveh, W. Xu, and A. Tannenbaum: Analysis and Design of Anisotropic Diffusion for Image Processing. In International Conference on Image Processing, volume II, pages 497–501, Nov. 1994.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Pierre Kornprobst
    • 1
    • 2
  • Rachid Deriche
    • 1
  • Gilles Aubert
    • 2
  1. 1.INRIASophia-Antipolis CedexFrance
  2. 2.Laboratoire J.A DieudonnéUMR no 6621 du CNRSNice-Cedex 2France

Personalised recommendations