Mumford and Shah Model and its Applications to Image Segmentation andImage Restoration

  • Leah Bar
  • Tony F. Chan
  • Ginmo Chung
  • Miyoun Jung
  • Nahum Kiryati
  • Rami Mohieddine
  • Nir Sochen
  • Luminita A. Vese


We present in this chapter an overview of the Mumford and Shah model for image segmentation. We discuss its various formulations, some of its properties, the mathematical framework, and several approximations. We also present numerical algorithms and segmentation results using the Ambrosio–Tortorelli phase-field approximations on one hand, and using the level set formulations on the other hand. Several applications of the Mumford–Shah problem to image restoration are also presented.


Image Restoration Impulse Noise Impulsive Noise Recovered Image Blur Kernel 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References and Further Reading

  1. 1.
    Adams RA (1975) Sobolev spaces. Academic, New YorkMATHGoogle Scholar
  2. 2.
    Alicandro R, Braides A, Shah J (1999) Free-discontinuity problems via functionals involving the L1-norm of the gradient and their approximation. Interfaces Free Bound 1:17–37MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Ambrosio L (1989) A compactness theorem for a special class of functions of bounded variation. Boll Un Mat Ital 3(B):857–881Google Scholar
  4. 4.
    Ambrosio L, Fusco N, Pallara D (2000) Functions of bounded variation and free discontinuity problems. Oxford University Press, New YorkMATHGoogle Scholar
  5. 5.
    Ambrosio L, Tortorelli VM (1990) Approximation of functionals depending on jumps by elliptic functionals via \(\Gamma \)-convergence. Comm Pure Appl Math 43(8):999–1036MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Ambrosio L, Tortorelli VM (1992) On the approximation of free discontinuity problems. Boll Un Mat Ital B7(6):105–123MathSciNetGoogle Scholar
  7. 7.
    Aubert G, Blanc-Féraud L, March R (2006) An approximation of the Mumford-Shah energy by a family of discrete edge-preserving functionals. Nonlinear Anal 64(9):1908–1930MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Aubert G, Kornprobst P (2006) Mathematical problems in image processing. Springer, New YorkMATHGoogle Scholar
  9. 9.
    Bar L, Brook A, Sochen N, Kiryati N (2007) Deblurring of color images corrupted by impulsive noise. IEEE Trans Image Process 16(4):1101–1111MathSciNetCrossRefGoogle Scholar
  10. 10.
    Bar L, Sochen N, Kiryati N (2004) Variational pairing of image segmentation and blind restoration. In Proceedings of 8th European conference on computer vision, vol 3022 of LNCS, pp 166–177Google Scholar
  11. 11.
    Bar L, Sochen N, Kiryati N (2005) Image deblurring in the presence of salt-and-pepper noise. In Proceedings of 5th international conference on scale space and PDE methods in computer vision, vol 3459 of LNCS, pp 107–118Google Scholar
  12. 12.
    Bar L, Sochen N, Kiryati N (2006) Image deblurring in the presence of impulsive noise. Int J Comput Vis 70:279–298CrossRefGoogle Scholar
  13. 13.
    Bar L, Sochen N, Kiryati N (2006) Semi-blind image restoration via Mumford-Shah regularization. IEEE Trans Image Process 15(2):483–493CrossRefGoogle Scholar
  14. 14.
    Bar L, Sochen N, Kiryati N (2007) Convergence of an iterative method for variational deconvolution and impulsive noise removal. SIAM J Multiscale Model Simulat 6:983–994MathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    Bar L, Sochen N, Kiryati N (2007) Restoration of images with piecewise space-variant blur. In Proceedings of 1st international conference on scale space and variational methods in computer vision, pp 533–544Google Scholar
  16. 16.
    Blake A, Zisserman A (1987) Visual reconstruction. MIT Press, CambridgeGoogle Scholar
  17. 17.
    Bourdin B (1999) Image segmentation with a finite element method. M2AN Math Model Numer Anal 33(2):229–244Google Scholar
  18. 18.
    Bourdin B, Chambolle A (2000) Implementation of an adaptive finite-element approximation of the Mumford-Shah functional. Numer Math 85(4):609–646MathSciNetMATHCrossRefGoogle Scholar
  19. 19.
    Braides A (1998) Approximation of free-discontinuity problems, vol 1694 of Lecture notes in mathematics. Springer, BerlinGoogle Scholar
  20. 20.
    Braides A, Dal Maso G (1997) Nonlocal approximation of the Mumford-Shah functional. Calc Var 5:293–322MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Bregman LM (1967) The relaxation method for finding common points of convex sets and its application to the solution of problems in convex programming. USSR Comp Math Phys 7:200–217CrossRefGoogle Scholar
  22. 22.
    Buades A, Coll B, Morel JM (2005) A review of image denoising algorithms, with a new one. SIAM MMS 4(2):490–530MathSciNetMATHGoogle Scholar
  23. 23.
    Chambolle A (1992) Un théorème de \(\gamma \)-convergence pour la segmentation des signaux. C R Acad Sci Paris Sér. I Math 314(3):191–196MathSciNetMATHGoogle Scholar
  24. 24.
    Chambolle A (1995) Image segmentation by variational methods: Mumford and Shah functional, and the discrete approximation. SIAM J Appl Math 55:827–863MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    Chambolle A (1999) Finite-differences discretizations of the Mumford-Shah functional. M2AN Math Model Numer Anal 33(2):261–288Google Scholar
  26. 26.
    Chambolle A (2000) Inverse problems in image processing and image segmentation: some mathematical and numerical aspects. In: Chidume CE (ed) ICTP Lecture notes series, vol 2. ICTPGoogle Scholar
  27. 27.
    Chambolle A, Dal Maso G (1999) Discrete approximation of the Mumford-Shah functional in dimension two. M2AN Math Model Numer Anal 33(4):651–672Google Scholar
  28. 28.
    Chan T, Vese L (1999) An active contour model without edges. Lecture Notes Comput Sci 1682:141–151CrossRefGoogle Scholar
  29. 29.
    Chan T, Vese L (2000) An efficient variational multiphase motion for the Mumford-Shah segmentation model. In 34th Asilomar conference on signals, systems, and computers, vol 1, pp 490–494Google Scholar
  30. 30.
    Chan T, Vese L (2001) Active contours without edges. IEEE Trans Image Process 10:266–277MATHCrossRefGoogle Scholar
  31. 31.
    Chan T, Vese L (2001) A level set algorithm for minimizing the Mumford-Shah functional in image processing. In IEEE/Computer Society proceedings of the 1st IEEE workshop on variational and level set methods in computer vision, pp 161–168Google Scholar
  32. 32.
    Chan TF, Shen J (2005) Image processing and analysis. Variational, PDE, wavelet, and stochastic methods. SIAM, PhiladelphiaGoogle Scholar
  33. 33.
    Chan TF, Wong CK (1998) Total variation blind deconvolution. IEEE Trans Image Process 7: 370–375CrossRefGoogle Scholar
  34. 34.
    Chung G, Vese LA (2005) Energy minimization based segmentation and denoising using a multilayer level set approach. Lecture Notes in Comput Sci 3757:439–455CrossRefGoogle Scholar
  35. 35.
    Chung G, Vese LA (2009) Image segmentation using a multilayer level-set approach. Computing Visual Sci 12(6):267–285MathSciNetMATHCrossRefGoogle Scholar
  36. 36.
    Cohen L, Bardinet E, Ayache N (1993) Surface reconstruction using active contour models. In SPIE ’93 conference on geometric methods in computer vision, San Diego, July 1993Google Scholar
  37. 37.
    Cohen LD (1997) Avoiding local minima for deformable curves in image analysis. In: Le Méhauté A, Rabut C, Schumaker LL (eds) Curves and Surfaces with applications in CAGD, pp 77–84Google Scholar
  38. 38.
    Dal Maso G (1993) An introduction to \(\Gamma \)-convergence. Progress in nonlinear differential equations and their applications. Birkhäuser, BostonGoogle Scholar
  39. 39.
    David G (2005) Singular sets of minimizers for the Mumford-Shah functional. Birkhäuser Verlag, BaselMATHGoogle Scholar
  40. 40.
    Evans LC (1998) Partial differential equations. American Mathematical Society, Providence, Rhode IslandMATHGoogle Scholar
  41. 41.
    Evans LC, Gariepy RF (1992) Measure theory and fine properties of functions. CRC PressGoogle Scholar
  42. 42.
    Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesean restoration of images. IEEE TPAMI 6:721–741MATHCrossRefGoogle Scholar
  43. 43.
    Gilboa G, Osher S (2007) Nonlocal linear image regularization and supervised segmentation. SIAM MMS 6(2):595–630MathSciNetMATHGoogle Scholar
  44. 44.
    Gilboa G, Osher S (2008) Nonlocal operators with applications to image processing. Multiscale Model Simulat 7(3):1005–1028MathSciNetMATHCrossRefGoogle Scholar
  45. 45.
    Huber PJ (1981) Robust statistics. Wiley, New YorkMATHCrossRefGoogle Scholar
  46. 46.
    Jung M, Chung G, Sundaramoorthi G, Vese LA, Yuille AL (2009) Sobolev gradients and joint variational image segmentation, denoising and deblurring. In: IS&T/SPIE on electronic imaging, vol 7246 of Computational imaging VII, pp 72460I–1–72460I–13Google Scholar
  47. 47.
    Jung M, Vese LA (2009) Nonlocal variational image deblurring models in the presence of gaussian or impulse noise. In: International conference on Scale Space and Variational Methods in Computer Vision (SSVM’ 09), vol 5567 of LNCS, pp 402–413Google Scholar
  48. 48.
    Kim J, Tsai A, Cetin M, Willsky AS (2002) A curve evolution-based variational approach to simultaneous image restoration and segmentation. In: Proceedings of IEEE international conference on image processing, vol 1, pp 109–112Google Scholar
  49. 49.
    Koepfler G, Lopez C, Morel JM (1994) A multiscale algorithm for image segmentation by variational methods. SIAM J Numer Anal 31(1): 282–299MathSciNetMATHCrossRefGoogle Scholar
  50. 50.
    Kundur D, Hatzinakos D (1996) Blind image deconvolution. Signal Process Mag 13:43–64CrossRefGoogle Scholar
  51. 51.
    Kundur D, Hatzinakos D (1996) Blind image deconvolution revisited. Signal Process Mag 13:61–63CrossRefGoogle Scholar
  52. 52.
    Larsen CJ (1998) A new proof of regularity for two-shaded image segmentations. Manuscripta Math 96:247–262MathSciNetMATHCrossRefGoogle Scholar
  53. 53.
    Leonardi GP, Tamanini I (1998) On minimizing partitions with infinitely many components. Ann Univ Ferrara - Sez. VII - Sc. Mat XLIV:41–57Google Scholar
  54. 54.
    Li C, Kao C-Y, Gore JC, Ding Z (2007) Implicit active contours driven by local binary fitting energy. In IEEE conference on computer vision and pattern recognition (CVPR), CVPR’07Google Scholar
  55. 55.
    Dal Maso G, Morel JM, Solimini S (1989) Variational approach in image processing – existence and approximation properties. C R Acad Sci Paris Sér. I Math 308(19):549–554MathSciNetMATHGoogle Scholar
  56. 56.
    Dal Maso G, Morel JM, Solimini S (1992) A variational method in image segmentation - existence and approximation properties. Acta Mathem 168(1–2):89–151MathSciNetMATHCrossRefGoogle Scholar
  57. 57.
    Massari U, Tamanini I (1993) On the finiteness of optimal partitions. Ann Univ Ferrara – Sez VII - Sc Mat XXXIX:167–185Google Scholar
  58. 58.
    Modica L (1987) The gradient theory of phase transitions and the minimal interface criterion. Arch Rational Mech Anal 98: 123–142MathSciNetMATHCrossRefGoogle Scholar
  59. 59.
    Modica L, Mortola S (1977) Un esempio di \(\gamma \)-convergenza. Boll Un Mat Ital B(5)(14): 285–299Google Scholar
  60. 60.
    Mohieddine R, Vese LA (2010) Open curve level set formulations for the Mumford and Shah segmentation model. UCLA C.A.M. Report, pp 10–33Google Scholar
  61. 61.
    Morel J-M, Solimini S (1995) Variational methods in image segmentation. Birkhäuser, BostonCrossRefGoogle Scholar
  62. 62.
    Mumford D, Shah J (1985) Boundary detection by minimizing functionals. In Proceedings of IEEE conference on computer vision and pattern recognition, pp 22–26Google Scholar
  63. 63.
    Mumford D, Shah J (1989) Boundary detection by minimizing functionals. In: Ullman S, Richards W (eds) Image understanding. Springer, Berlin, pp 19–43Google Scholar
  64. 64.
    Mumford D, Shah J (1989) Optimal approximations by piecewise smooth functions and associated variational problems. Comm Pure Appl Math 42:577–685MathSciNetMATHCrossRefGoogle Scholar
  65. 65.
    Neuberger JW (1997) Sobolev gradients and differential equations. Springer lecture notes in mathematics, vol 1670Google Scholar
  66. 66.
    Nikolova M (2002) Minimizers of cost-functions involving nonsmooth data-fidelity terms: application to the processing of outliers. SIAM J Numer Anal 40:965–994MathSciNetMATHCrossRefGoogle Scholar
  67. 67.
    Nikolova M (2004) A variational approach to remove outliers and impulse noise. J Math Imaging Vis 20:99–120MathSciNetCrossRefGoogle Scholar
  68. 68.
    Osher S, Burger M, Goldfarb D, Xu J, Yin W (2005) An iterative regularization method for total variation based image restoration. SIAM MMS 4:460–489MathSciNetMATHGoogle Scholar
  69. 69.
    Osher S, Sethian JA (1988) Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulation. J Comput Phys 79:12–49MathSciNetMATHCrossRefGoogle Scholar
  70. 70.
    Osher SJ, Fedkiw RP (2002) Level set methods and dynamic implicit surfaces. Springer, New YorkGoogle Scholar
  71. 71.
    Renka RJ (2006) A Simple Explanation of the Sobolev Gradient Method. (online manuscript at
  72. 72.
    Richardson WB (2008) Sobolev gradient preconditioning for image processing PDEs. Commun Numer Meth Eng 24:493–504MATHCrossRefGoogle Scholar
  73. 73.
    Rudin L, Osher S (1994) Total variation based image restoration with free local constraints. In: Proceedings of IEEE international conference on image processing, vol 1, Austin, pp 31–35Google Scholar
  74. 74.
    Rudin LI, Osher S, Fatemi E (1992) Non linear total variation based noise removal algorithms. Physica D 60:259–268MATHCrossRefGoogle Scholar
  75. 75.
    Samson C, Blanc-Féraud L, Aubert G, Zerubia J (1999) Multiphase evolution and variational image classification. Technical Report 3662, INRIA Sophia AntipolisGoogle Scholar
  76. 76.
    Sethian JA (1996) Level set methods. Evolving interfaces in geometry, fluid mechanics, computer vision, and materials science. Cambridge University PressGoogle Scholar
  77. 77.
    Sethian JA (1999) Level set methods and fast marching methods. Evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials science. Cambridge University Press, CambridgeGoogle Scholar
  78. 78.
    Shah J (1996) A common framework for curve evolution, segmentation and anisotropic diffusion. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 136–142Google Scholar
  79. 79.
    Smereka P (2000) Spiral crystal growth. Physica D 138:282–301MathSciNetMATHCrossRefGoogle Scholar
  80. 80.
    Tamanini I (1996) Optimal approximation by piecewise constant functions. In: Progress in nonlinear differential equations and their applications, vol 25. Birkhäuser Verlag, Basel, pp 73–85Google Scholar
  81. 81.
    Tamanini I, Congedo G (1996) Optimal segmentation of unbounded functions. Rend Sem Mat Univ Padova 95:153–174MathSciNetMATHGoogle Scholar
  82. 82.
    Tikhonov AN, Arsenin V (1977) Solutions of ill-posed problems. Winston, WashingtonMATHGoogle Scholar
  83. 83.
    Tsai A, Yezzi A, Willsky A (2001) Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolation, and magnification. IEEE Trans Image Process 10(8):1169–1186MATHCrossRefGoogle Scholar
  84. 84.
    Vese LA, Chan TF (2002) A multiphase level set framework for image segmentation using the Mumford and Shah model. Int J Comput Vis 50(3):271–293MATHCrossRefGoogle Scholar
  85. 85.
    Vogel CR, Oman ME (1998) Fast, robust total variation-based reconstruction of noisy, blurred images. IEEE Trans Image Process 7:813–824MathSciNetMATHCrossRefGoogle Scholar
  86. 86.
    Weisstein EW Minimal residual method. MathWorld–A Wolfram Web Resource.
  87. 87.
    You Y, Kaveh M (1996) A regularization approach to joint blur identification and image restoration. IEEE Trans Image Process 5:416–428CrossRefGoogle Scholar
  88. 88.
    Zhao HK, Chan T, Merriman B, Osher S (1996) A variational level set approach to multiphase motion. J Comput Phys 127:179–195MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Leah Bar
    • 1
  • Tony F. Chan
    • 2
  • Ginmo Chung
    • 3
  • Miyoun Jung
    • 4
  • Nahum Kiryati
    • 5
  • Rami Mohieddine
    • 6
  • Nir Sochen
    • 7
  • Luminita A. Vese
    • 8
  1. 1.University of MinnesotaMinnesotaUSA
  2. 2.University of California Los AngelesLos AngelesUSA
  3. 3.Nanyang Technological UniversitySingaporeSingapore
  4. 4.University of California Los AngelesLos AngelesUSA
  5. 5.Tel Aviv UniversityTel AvivIsrael
  6. 6.University of California Los AngelesLos AngelesUSA
  7. 7.Tel Aviv UniversityTel AvivIsrael
  8. 8.University of California Los AngelesLos AngelesUSA

Personalised recommendations