Energy Minimization Methods

  • Mila Nikolova


Energy minimization methods are a very popular tool in image and signalprocessing. This chapter deals with images defined on a discrete finite set. Energyminimization methods are presented from a nonclassical standpoint: weprovide analytical results on their minimizers that reveal salient featuresof the images recovered in this way, as a function of the shape of theenergy itself. The energies under consideration can be differentiable ornot, convex or not. Examples and illustrations corroborate the presentedresults. Applications that take benefit from these results are presented as well.


Global Minimizer Noisy Data Homogeneous Region Impulse Noise Shrinkage Estimator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References and Further Reading

  1. 1.
    Alliney S (1992) Digital filters as absolute norm regularizers. IEEE Trans Signal Process SP-40:1548–1562Google Scholar
  2. 2.
    Alter F, Durand S, Forment J (2005) Adapted total variation for artifact free decompression of JPEG images. J Math Imaging Vis 23: 199–211CrossRefGoogle Scholar
  3. 3.
    Ambrosio L, Fusco N, Pallara D (2000) Functions of bounded variation and free discontinuity Problems. Oxford Mathematical Monographs, Oxford University PressGoogle Scholar
  4. 4.
    Antoniadis A, Fan J (2001) Regularization of wavelet approximations. J Acoust Soc Am 96: 939–967MathSciNetMATHGoogle Scholar
  5. 5.
    Aubert G, Kornprobst P (2006) Mathematical problems in image processing, 2nd edn. Springer, BerlinMATHGoogle Scholar
  6. 6.
    Aujol J-F, Gilboa G, Chan T, Osher S (2006) Structure-texture image decomposition - modeling, algorithms, and parameter selection. Int J Comput Vis 67:111–136CrossRefGoogle Scholar
  7. 7.
    Bar L, Brook A, Sochen N, Kiryati N (2007) Deblurring of color images corrupted by impulsive noise. IEEE Trans Image Process 16:1101–1111MathSciNetCrossRefGoogle Scholar
  8. 8.
    Bar L, Kiryati N, Sochen N (2006) Image deblurring in the presence of impulsive noise, International. J Comput Vision 70:279–298CrossRefGoogle Scholar
  9. 9.
    Bar L, Sochen N, Kiryati N (2005) Image deblurring in the presence of salt-and-pepper noise. In Proceeding of 5th international conference on scale space and PDE methods in computer vision, ser LNCS, vol 3439, pp 107–118Google Scholar
  10. 10.
    Belge M, Kilmer M, Miller E (2000) Wavelet domain image restoration with adaptive edge-preserving regularization. IEEE Trans Image Process 9:597–608MATHCrossRefGoogle Scholar
  11. 11.
    Besag JE (1974) Spatial interaction and the statistical analysis of lattice systems (with discussion). J Roy Stat Soc B 36:192–236MathSciNetMATHGoogle Scholar
  12. 12.
    Besag JE (1989) Digital image processing : towards Bayesian image analysis. J Appl Stat 16:395–407CrossRefGoogle Scholar
  13. 13.
    Black M, Rangarajan A (1996) On the unification of line processes, outlier rejection, and robust statistics with applications to early vision. Int J Comput Vis 19:57–91CrossRefGoogle Scholar
  14. 14.
    Blake A, Zisserman A (1987) Visual reconstruction. MIT Press, CambridgeGoogle Scholar
  15. 15.
    Bloomfield B, Steiger WL (1983) Least absolute deviations: theory, applications and algorithms. Birkhäuser, BostonMATHGoogle Scholar
  16. 16.
    Bobichon Y, Bijaoui A (1997) Regularized multiresolution methods for astronomical image enhancement. Exp Astron 7:239–255CrossRefGoogle Scholar
  17. 17.
    Bouman C, Sauer K (1993) A generalized Gaussian image model for edge-preserving map estimation. IEEE Trans Image Process 2:296–310CrossRefGoogle Scholar
  18. 18.
    Bouman C, Sauer K (1996) A unified approach to statistical tomography using coordinate descent optimization. IEEE Trans Image Process 5:480–492CrossRefGoogle Scholar
  19. 19.
    Bredies K, Kunich K, Pock T (2010) Total generalized variation. SIAM J Imaging Sci (to appear)Google Scholar
  20. 20.
    Candès EJ, Donoho D, Ying L (2005) Fast discrete curvelet transforms. SIAM Multiscale Model Simul 5:861–899CrossRefGoogle Scholar
  21. 21.
    Candès EJ, Guo F (2002) New multiscale transforms, minimum total variation synthesis. Applications to edge-preserving image reconstruction. Signal Process 82:1519–1543MATHCrossRefGoogle Scholar
  22. 22.
    Catte F, Coll T, Lions PL, Morel JM (1992) Image selective smoothing and edge detection by nonlinear diffusion (I). SIAM J Num Anal 29:182–193MathSciNetMATHCrossRefGoogle Scholar
  23. 23.
    Chambolle A (2004) An algorithm for total variation minimization and application. J Math Imaging Vis 20:89–97MathSciNetCrossRefGoogle Scholar
  24. 24.
    Chambolle A, Lions P-L (1997) Image recovery via total variation minimization and related problems. Numer Math 76:167–188MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    Chan T, Esedoglu S (2005) Aspects of total variationregularized l 1function approximation. SIAM J Appl Math 65:1817–1837MathSciNetMATHCrossRefGoogle Scholar
  26. 26.
    Chan TF, Wong CK (1998) Total variation blind deconvolution. IEEE Trans Image Process 7:370–375CrossRefGoogle Scholar
  27. 27.
    Charbonnier P, Blanc-Féraud L, Aubert G, Barlaud M (1997) Deterministic edge-preserving regularization in computed imaging. IEEE Trans Image Process 6:298–311CrossRefGoogle Scholar
  28. 28.
    Chellapa R, Jain A (1993) Markov random fields: theory and application. Academic, BostonGoogle Scholar
  29. 29.
    Chesneau C, Fadili J, Starck J-L (2008) Stein block thresholding for image denoising. Technical reportGoogle Scholar
  30. 30.
    Ciarlet PG (1989) Introduction to numerical linear algebra and optimization. Cambridge University Press, CambridgeGoogle Scholar
  31. 31.
    Coifman RR, Sowa A (2000) Combining the calculus of variations and wavelets for image enhancement. Appl Comput Harmon Anal 9: 1–18MathSciNetMATHCrossRefGoogle Scholar
  32. 32.
    Demoment G (1989) Image reconstruction and restoration: overview of common estimation structure and problems. IEEE Trans Acoust Speech Signal Process ASSP-37:2024–2036CrossRefGoogle Scholar
  33. 33.
    Do MN, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 15:1916–1933MathSciNetGoogle Scholar
  34. 34.
    Dobson D, Santosa F (1996) Recovery of blocky images from noisy and blurred data. SIAM J Appl Math 56:1181–1199MathSciNetMATHCrossRefGoogle Scholar
  35. 35.
    Donoho DL, Johnstone IM (1994) Ideal spatial adaptation by wavelet shrinkage. Biometrika 81:425–455MathSciNetMATHCrossRefGoogle Scholar
  36. 36.
    Donoho DL, Johnstone IM (1995) Adapting to unknown smoothness via wavelet shrinkage. J Acoust Soc Am 90:1200–1224MathSciNetMATHGoogle Scholar
  37. 37.
    Dontchev AL, Zollezi T (1993) Well-posed optimization problems. Springer, New YorkMATHGoogle Scholar
  38. 38.
    Durand S, Froment J (2003) Reconstruction of wavelet coefficients using total variation minimization. SIAM J Sci Comput 24:1754–1767MathSciNetMATHCrossRefGoogle Scholar
  39. 39.
    Durand S, Nikolova M (2006) Stability of minimizers of regularized least squares objective functions I: study of the local behavior. Appl Math Optim (Springer, New York) 53:185–208MathSciNetMATHGoogle Scholar
  40. 40.
    Durand S, Nikolova M (2006) Stability of minimizers of regularized least squares objective functions II: study of the global behaviour. Appl Math Optim (Springer, New York) 53:259–277MathSciNetMATHGoogle Scholar
  41. 41.
    Durand S, Nikolova M (2007) Denoising of frame coefficients using \({\mathcal{l}}^{1}\) data-fidelity term and edge-preserving regularization. SIAM J Multiscale Model Simulat 6:547–576Google Scholar
  42. 42.
    Duval V, Aujol J-F, Gousseau Y (2009) The TVL1 model: a geometric point of view. SIAM J Multiscale Model Simulat 8:154–189MathSciNetMATHCrossRefGoogle Scholar
  43. 43.
    Ekeland I, Temam R (1976) Convex analysis and variational problems. North-Holland/SIAM, AmsterdamMATHGoogle Scholar
  44. 44.
    Fessler F (1996) Mean and variance of implicitly defined biased estimators (such as penalized maximum likelihood): applications to tomography. IEEE Trans Image Process 5:493–506CrossRefGoogle Scholar
  45. 45.
    Fiacco A, McCormic G (1990) Nonlinear programming. Classics in applied mathematics. SIAM, PhiladelphiaGoogle Scholar
  46. 46.
    Froment J, Durand S (2001) Artifact free signal denoising with wavelets. In Proceedings of the IEEE international conference on acoustics, speech and signal processing, vol 6Google Scholar
  47. 47.
    Geman D (1990) Random fields and inverse problems in imaging, vol 1427, École d’Été de Probabilités de. Saint-Flour XVIII - 1988, Springer, lecture notes in mathematics ed., pp 117–193Google Scholar
  48. 48.
    Geman D, Reynolds G (1992) Constrained restoration and recovery of discontinuities. IEEE Trans Pattern Anal Mach Intell PAMI-14: 367–383Google Scholar
  49. 49.
    Geman D, Yang C (1995) Nonlinear image recovery with half-quadratic regularization. IEEE Trans Image Process IP-4:932–946Google Scholar
  50. 50.
    Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell PAMI-6:721–741Google Scholar
  51. 51.
    Golub G, Van Loan C (1996) Matrix computations, 3rd edn. Johns Hopkins University Press, BaltimoreMATHGoogle Scholar
  52. 52.
    Green PJ (1990) Bayesian reconstructions from emissiontomography data using a modified em algorithm. IEEE Trans Med Imaging MI-9:84–93Google Scholar
  53. 53.
    Haddad A, Meyer Y (2007) Variational methods in image processing, in “Perspective in Nonlinear Partial Differential equations in Honor of Haïm Brezis,” Contemp Math (AMS) 446:273–295MathSciNetCrossRefGoogle Scholar
  54. 54.
    Herman G (1980) Image reconstruction from projections. The fundamentals of computerized tomography. Academic, New YorkMATHGoogle Scholar
  55. 55.
    Hiriart-Urruty J-B, Lemaréchal C (1996) Convex analysis and minimization algorithms, vols I, II. Springer, BerlinGoogle Scholar
  56. 56.
    Hofmann B (1986) Regularization for applied inverse and ill posed problems. Teubner, LeipzigMATHGoogle Scholar
  57. 57.
    Kailath T (1974) A view of three decades of linear filtering theory. IEEE Trans Inf Theory IT-20:146–181Google Scholar
  58. 58.
    Kak A, Slaney M (1987) Principles of computerized tomographic imaging. IEEE Press, New YorkGoogle Scholar
  59. 59.
    Katsaggelos AKE (1991) Digital image restoration. Springer, New YorkCrossRefGoogle Scholar
  60. 60.
    Keren D, Werman M (1993) Probabilistic analysis of regularization. IEEE Trans Pattern Anal Mach Intell PAMI-15:982–995Google Scholar
  61. 61.
    Lange K (1990) Convergence of EM image reconstruction algorithms with Gibbs priors. IEEE Trans Med Imaging 9:439–446CrossRefGoogle Scholar
  62. 62.
    Li S (1995) Markov random field modeling in computer vision, 1st edn. Springer, New YorkGoogle Scholar
  63. 63.
    Li SZ (1995) On discontinuity-adaptive smoothness priors in computer vision. IEEE Trans Pattern Anal Mach Intell PAMI-17:576–586Google Scholar
  64. 64.
    Luisier F, Blu T (2008) SURE-LET multichannel image denoising: interscale orthonormal wavelet thresholding. IEEE Trans Image Process 17:482–492MathSciNetCrossRefGoogle Scholar
  65. 65.
    Malgouyres F (2002) Minimizing the total variation under a general convex constraint for image restoration. IEEE Trans Image Process 11:1450–1456MathSciNetCrossRefGoogle Scholar
  66. 66.
    Morel J-M, Solimini S (1995) Variational methods in image segmentation. Birkhäuser, BaselCrossRefGoogle Scholar
  67. 67.
    Morozov VA (1993) Regularization methods for ill posed problems. CRC Press, Boca RatonMATHGoogle Scholar
  68. 68.
    Moulin P, Liu J (1999) Analysis of multiresolution image denoising schemes using generalized Gaussian and complexity priors. IEEE Trans Image Process 45:909–919MathSciNetMATHGoogle Scholar
  69. 69.
    Moulin P, Liu J (2000) Statistical imaging and complexity regularization. IEEE Trans Inf Theory 46:1762–1777MathSciNetMATHCrossRefGoogle Scholar
  70. 70.
    Mumford D, Shah J (1989) Optimal approximations by piecewise smooth functions and associated variational problems. Commun Pure Appl Math 42:577–684MathSciNetMATHCrossRefGoogle Scholar
  71. 71.
    Nashed M, Scherzer O (1998) Least squares and bounded variation regularization with nondifferentiable functional. Numer Funct Anal Optim 19:873–901MathSciNetMATHCrossRefGoogle Scholar
  72. 72.
    Nikolova M (1996) Regularisation functions and estimators. In Proceedings of the IEEE international conference on image processing, vol 2, pp 457–460CrossRefGoogle Scholar
  73. 73.
    Nikolova M (1997) Estimées localement fortement homogènes. Comptes-Rendus de l’Acad émie des Sciences 325 (série 1):665–670Google Scholar
  74. 74.
    Nikolova M (2000) Thresholding implied by truncated quadratic regularization. IEEE Trans Image Process 48:3437–3450MathSciNetMATHGoogle Scholar
  75. 75.
    Nikolova M (2001) Image restoration by minimizing objective functions with non-smooth data-fidelity terms. In IEEE international conference on computer vision/workshop on variational and level-set methods, pp 11–18CrossRefGoogle Scholar
  76. 76.
    Nikolova M (2002) Minimizers of cost-functions involving nonsmooth data-fidelity terms. Application to the processing of outliers. SIAM J Num Anal 40:965–994MathSciNetMATHCrossRefGoogle Scholar
  77. 77.
    Nikolova M (2004) A variational approach to remove outliers and impulse noise. J Math Imaging Vis 20:99–120MathSciNetCrossRefGoogle Scholar
  78. 78.
    Nikolova M (2004) Weakly constrained minimization. Application to the estimation of images and signals involving constant regions. J Math Imaging Vis 21:155–175MathSciNetCrossRefGoogle Scholar
  79. 79.
    Nikolova M (2005) Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. SIAM J Multiscale Model Simulat 4:960–991MathSciNetMATHCrossRefGoogle Scholar
  80. 80.
    Nikolova M (2007) Analytical bounds on the minimizers of (nonconvex) regularized least- squares. AIMS J Inverse Probl Imaging 1: 661–677MathSciNetCrossRefGoogle Scholar
  81. 81.
    Nikolova M (2009) Semi-explicit solution and fast minimization scheme for an energy with \({\mathcal{l}}^{1}\)-fitting and Tikhonov-like regularization. J Math Imaging Vis 34:32–47Google Scholar
  82. 82.
    Perona P, Malik J (1990) Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell PAMI-12:629–639Google Scholar
  83. 83.
    Pham TT, De Figueiredo RJP (1989) Maximum likelihood estimation of a class of non-Gaussian densities with application to \({l}_{\mathrm{p}}\) deconvolution. IEEE Trans Signal Process 37:73–82Google Scholar
  84. 84.
    Rice JR, White JS (1964) Norms for smoothing and estimation. SIAM Rev 6:243–256MathSciNetMATHCrossRefGoogle Scholar
  85. 85.
    Rockafellar RT, Wets JB (1997) Variational analysis. Springer, New YorkGoogle Scholar
  86. 86.
    Rudin L, Osher S, Fatemi C (1992) Nonlinear total variation based noise removal algorithm. Physica 60 D:259–268Google Scholar
  87. 87.
    Sauer K, Bouman C (1993) A local update strategy for iterative reconstruction from projections. IEEE Trans Signal Process SP-41:534–548Google Scholar
  88. 88.
    Scherzer O, Grasmair M, Grossauer H, Haltmeier M, Lenzen F (2009) Variational problems in imaging. Springer, New YorkGoogle Scholar
  89. 89.
    Simoncelli EP, Adelson EH (Sept 1996) Noise removal via Bayesian wavelet coding. In Proceedings of the IEEE international conference on image processing, Lausanne, Switzerland, pp 379–382CrossRefGoogle Scholar
  90. 90.
    Stevenson R, Delp E (1990) Fitting curves with discontinuities. In Proceedings of the 1st international workshop on robust computer vision, Seattle, WA, pp 127–136Google Scholar
  91. 91.
    Tautenhahn U (1994) Error estimates for regularized solutions of non-linear ill posed problems. Inverse Probl 10:485–500MathSciNetMATHCrossRefGoogle Scholar
  92. 92.
    Teboul S, Blanc-Féraud L, Aubert G, Barlaud M (1998) Variational approach for edge-preserving regularization using coupled PDE’s. IEEE Trans Image Process 7:387–397CrossRefGoogle Scholar
  93. 93.
    Tikhonov A, Arsenin V (1977) Solutions of ill posed problems, Winston, WashingtonMATHGoogle Scholar
  94. 94.
    Vogel C (2002) Computational methods for inverse problems. Frontiers in applied mathematics series, vol 23. SIAM, New YorkMATHCrossRefGoogle Scholar
  95. 95.
    Welk M, Steidl G, Weickert J (2008) Locally analytic schemes: a link between diffusion filtering and wavelet shrinkage. Appl Comput Harmon Anal 24:195–224MathSciNetMATHCrossRefGoogle Scholar
  96. 96.
    Winkler G (2006) Image analysis, random fields and Markov chain Monte Carlo methods. A mathematical introduction. Applications of mathematics, 2nd edn, vol 27. Stochastic models and applied probability. Springer, BerlinGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Mila Nikolova
    • 1
  1. 1.ENS Cachan, CNRS UniversSudCachan CedexFrance

Personalised recommendations