We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

EM Algorithms | SpringerLink
Skip to main content
SpringerLink
Log in

EM Algorithms

  • Reference work entry
Handbook of Mathematical Methods in Imaging

Abstract

Expectation-maximization algorithms, or em algorithms for short, are iterative algorithms designed to solve maximum likelihood estimation problems. The general setting is that one observes a random sample Y 1, Y 2, …, Y n of a random variable Y whose probability density function (pdf) \(f(\,\cdot \,\vert \,x_{o})\) with respect to some (known) dominating measure is known up to an unknown “parameter” x o . The goal is to estimate x o and, one might add, to do it well. In this chapter, that means to solve the maximum likelihood problem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 1,200.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 549.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aronszajn, N., Smith, K.T.: Theory of Bessel potentials. I. Ann. Inst. Fourier (Grenoble) 11, 385–475 (1961). www.numdam.org

  2. Atkinson, K.E.: The numerical solution of integral equations on the half line. SIAM J. Numer. Anal. 6, 375–397 (1969)

    Article  MATH  MathSciNet  Google Scholar 

  3. Bardsley, J.M., Luttman, A.: Total variation-penalized Poisson likelihood estimation for ill-posed problems. Adv. Comput. Math. 31, 35–39 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  4. Bertero, M., Bocacci, P., Desiderá, G., Vicidomini, G.: Image de-blurring with Poisson data: from cells to galaxies. Inverse Probl. 25(123006), 26 (2009)

    Google Scholar 

  5. Browne, J., De Pierro, A.R.: A row-action alternative to the EM algorithm for maximizing likelihoods in emission tomography. IEEE Trans. Med. Imaging 15, 687–699 (1996)

    Article  Google Scholar 

  6. Brune, C., Sawatzky, A., Burger, M.: Bregman-EM-TV methods with application to optical nanoscopy. In: Second International Conference on Scale Space and Variational Methods in Computer Vision, Voss. Lecture Notes in Computer Science, vol. 5567, pp. 235–246. Springer, Berlin (2009)

    Google Scholar 

  7. Byrne, C.L.: Iterative image reconstruction algorithms based on cross-entropy minimization. IEEE Trans. Image Process. 2, 96–103 (1993)

    Article  Google Scholar 

  8. Byrne, C.L.: Block-iterative methods for image reconstruction from projections. IEEE Trans. Image Process. 5, 792–794 (1996)

    Article  Google Scholar 

  9. Byrne, C.L.: Accelerating the EMML algorithm and related iterative algorithms by rescaled block-iterative methods. IEEE Trans. Image Process. 7, 792–794 (1998)

    Article  MathSciNet  Google Scholar 

  10. Byrne, C.L.: Likelihood maximization for list-mode emission tomographic image reconstruction. IEEE Trans. Med. Imaging 20, 1084–1092 (2001)

    Article  Google Scholar 

  11. Byrne, C.L.: Choosing parameters in block-iterative or ordered subset reconstruction algorithms. IEEE Trans. Image Process. 14, 321–327 (2005)

    Article  MathSciNet  Google Scholar 

  12. Byrne, C.L.: Signal Processing: A Mathematical Approach. AK Peters, Wellesley (2005)

    Google Scholar 

  13. Byrne, C.L.: Applied Iterative Methods. AK Peters, Wellesley (2008)

    MATH  Google Scholar 

  14. Byrne, C.L., Fiddy, M.A.: Images as power spectra; reconstruction as a Wiener filter approximation. Inverse Probl. 4, 399–409 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  15. Cao, Y.u., Eggermont, P.P.B., Terebey, S.: Cross Burg entropy maximization and its application to ringing suppression in image reconstruction. IEEE Trans. Image Process. 8, 286–292 (1999)

    Google Scholar 

  16. Censor, Y., Eggermont, P.P.B., Gordon, D.: Strong under relaxation in Kaczmarz’s method for inconsistent systems. Numer. Math. 41, 83–92 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  17. Censor, Y., Lent, A.H.: Optimization of “log x” entropy over linear equality constraints. SIAM J. Control. Optim. 25, 921–933 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  18. Censor, Y., Segman, J.: On block-iterative entropy maximization. J. Inf. Optim. Sci. 8, 275–291 (1987)

    MATH  MathSciNet  Google Scholar 

  19. Censor, Y., Zenios, S.A.: Proximal minimization algorithm with D-functions. J. Optim. Theory Appl. 73, 451–464 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  20. Cover, T.M.: An algorithm for maximizing expected log investment return. IEEE Trans. Inf. Theory 30, 369–373 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  21. Crowther, R.A., DeRosier, D.J., Klug, A.: The reconstruction of three-dimensional structure from projections and its application to electron microscopy. Proc. R. Soc. Lond. A Math. Phys. Sci. 317(3), 19–340 (1971)

    Google Scholar 

  22. Csiszár, I.: I-divergence geometry of probability distributions and minimization problems. Ann. Probab. 3, 146–158 (1975)

    Article  MATH  Google Scholar 

  23. Csiszár, I., Tusnády, G.: Information geometry and alternating minimization procedures. Stat. Decis. 1(Supplement 1), 205–237 (1984)

    Google Scholar 

  24. Daley, D.J., Vere-Jones, D.: An Introduction to the Theory of Point Processes. Springer, New York (2003)

    MATH  Google Scholar 

  25. Darroch, J.N., Ratcliff, D.: Generalized iterative scaling for log-linear models. Ann. Math. Stat. 43, 1470–1480 (1972)

    Article  MATH  MathSciNet  Google Scholar 

  26. Daube-Witherspoon, M.E., Muehllehner, G.: An iterative space reconstruction algorithm suitable for volume ECT. IEEE Trans. Med. Imaging 5, 61–66 (1986)

    Article  Google Scholar 

  27. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 37, 1–38 (1977)

    MathSciNet  Google Scholar 

  28. De Pierro, A.R.: On the convergence of the iterative image space reconstruction algorithm for volume ECT. IEEE Trans. Med. Imaging 6, 174–175 (1987)

    Article  Google Scholar 

  29. De Pierro, A.R.: A modified expectation maximization algorithm for penalized likelihood estimation in emission tomography. IEEE Trans. Med. Imaging 14, 132–137 (1995)

    Article  Google Scholar 

  30. De Pierro, A., Yamaguchi, M.: Fast EM-like methods for maximum a posteriori estimates in emission tomography. Trans. Med. Imaging 20, 280–288 (2001)

    Article  Google Scholar 

  31. Dey, N., Blanc-Ferraud, L., Zimmer, Ch., Roux, P., Kam, Z., Olivo-Martin, J.-Ch., Zerubia, J.: Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution. Microsc. Res. Tech. 69, 260–266 (2006)

    Article  Google Scholar 

  32. Duijster, A., Scheunders, P., De Backer, S.: Wavelet-based EM algorithm for multispectral-image restoration. IEEE Trans. Geosci. Remote Sens. 47, 3892–3898 (2009)

    Article  Google Scholar 

  33. Eggermont, P.P.B.: Multiplicative iterative algorithms for convex programming. Linear Algebra Appl. 130, 25–42 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  34. Eggermont, P.P.B.: Nonlinear smoothing and the EM algorithm for positive integral equations of the first kind. Appl. Math. Optim. 39, 75–91 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  35. Eggermont, P.P.B., Herman, G.T., Lent, A.H.: Iterative algorithms for large partitioned linear systems with applications to image reconstruction. Linear Algebra Appl. 40, 37–67 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  36. Eggermont, P.P.B., LaRiccia, V.N.: Smoothed maximum likelihood density estimation for inverse problems. Ann. Stat. 23, 199–220 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  37. Eggermont, P.P.B., LaRiccia, V.N.: Maximum penalized likelihood estimation and smoothed EM algorithms for positive integral equations of the first kind. Numer. Funct. Anal. Optim. 17, 737–754 (1997)

    Article  MathSciNet  Google Scholar 

  38. Eggermont, P.P.B., LaRiccia, V.N.: On EM-like algorithms for minimum distance estimation. Manuscript, University of Delaware (1998)

    Google Scholar 

  39. Eggermont, P.P.B., LaRiccia, V.N.: Maximum Penalized Likelihood Estimation, I: Density Estimation. Springer, New York (2001)

    Google Scholar 

  40. Elfving, T.: On some methods for entropy maximization and matrix scaling. Linear Algebra Appl. 34, 321–339 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  41. Fessler, J.A., Ficaro, E.P., Clinthorne, N.H., Lange, K.: Grouped coordinate ascent algorithms for penalized log-likelihood transmission image reconstruction. IEEE Trans. Med. Imaging 16, 166–175 (1997)

    Article  Google Scholar 

  42. Fessler, J.A., Hero, A.O.: Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Trans. Image Process. 4, 1417–1429 (1995)

    Article  Google Scholar 

  43. Figueiredo, M.A.T., Nowak, R.D.: An EM algorithm for wavelet-based image restoration. IEEE Trans. Image Process. 12, 906–916 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  44. Frank, J.: Three-Dimensional Electron Microscopy of Macromolecular Assemblies, 2nd edn. Oxford University Press, New York (2006)

    Book  Google Scholar 

  45. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)

    Article  MATH  Google Scholar 

  46. Geman, S., McClure, D.E.: Bayesian image analysis, an application to single photon emission tomography. In: Proceedings of the Statistical Computing Section, Las Vegas, pp. 12–18. American Statistical Association (1985)

    Google Scholar 

  47. Good, I.J.: A nonparametric roughness penalty for probability densities. Nature 229, 29–30 (1971)

    Google Scholar 

  48. Gordon, R., Bender, R., Herman, G.T.: Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography. J. Theor. Biol. 29, 471–482 (1970)

    Article  Google Scholar 

  49. Green, P.J.: Bayesian reconstructions from emission tomography data using a modified EM algorithm. IEEE Trans. Med. Imaging 9, 84–93 (1990)

    Article  Google Scholar 

  50. Guillaume, M., Melon, P., Réfrégier, P.: Maximum-likelihood estimation of an astronomical image from a sequence at low photon levels. J. Opt. Soc. Am. A 15, 2841–2848 (1998)

    Article  Google Scholar 

  51. Haltmeier, M., Leitão, A., Resmerita, E.: On regularization methods of EM-Kaczmarz type. Inverse Probl. 25(075008), 17 (2009)

    Google Scholar 

  52. Hanke, M.: Accelerated Landweber iterations for the solution of ill-posed problems. Numer. Math. 60, 341–373 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  53. Hartley, H.O.: Maximum likelihood estimation from incomplete data. Biometrics 14, 174–194 (1958)

    Article  MATH  Google Scholar 

  54. Hebert, T., Leahy, R.: A generalized EM algorithm for 3-D Bayesian reconstruction from Poisson data using Gibbs priors. IEEE Trans. Med. Imaging 8, 194–202 (1989)

    Article  Google Scholar 

  55. Herman, G.T.: Fundamentals of Computerized Tomography: Image Reconstruction from Projections. Springer, New York (2009)

    Book  Google Scholar 

  56. Herman, G.T., Meyer, L.B.: Algebraic reconstruction techniques can be made computationally efficient. IEEE Trans. Med. Imaging 12, 600–609 (1993)

    Article  Google Scholar 

  57. Holte, S., Schmidlin, P., Lindén, A., Rosenqvist, G., Eriksson, L.: Iterative image reconstruction for positron emission tomography: a study of convergence and quantitation problems. IEEE Trans. Nucl. Sci. 37, 629–635 (1990)

    Article  Google Scholar 

  58. Horváth, I., Bagoly, Z., Balász, L.G., de Ugarte Postigo, A., Veres, P., Mészáros, A.: Detailed classification of Swift’s Gamma-ray bursts. J. Astrophys. 713, 552–557 (2010)

    Article  Google Scholar 

  59. Hudson, H.M., Larkin, R.S.: Accelerated image reconstruction using ordered subsets of projection data. IEEE Trans. Med. Imaging 13, 601–609 (1994)

    Article  Google Scholar 

  60. Kamphuis, C., Beekman, F.J., Viergever, M.A.: Evaluation of OS-EM vs. EM-ML for 1D, 2D and fully 3D SPECT reconstruction. IEEE Trans. Nucl. Sci. 43, 2018–2024 (1996)

    Google Scholar 

  61. Kondor, A.: Method of convergent weights – an iterative procedure for solving Fredholm’s integral equations of the first kind. Nucl. Instrum. Methods 216, 177–181 (1983)

    Article  Google Scholar 

  62. Lange, K.: Convergence of EM image reconstruction algorithms with Gibbs smoothing. IEEE Trans. Med. Imaging 9, 439–446 (1990)

    Article  Google Scholar 

  63. Lange, K., Bahn, M., Little, R.: A theoretical study of some maximum likelihood algorithms for emission and transmission tomography. IEEE Trans. Med. Imaging 6, 106–114 (1987)

    Article  Google Scholar 

  64. Lange, K., Carson, R.: EM reconstruction algorithms for emission and transmission tomography. J. Comput. Assist. Tomogr. 8, 306–316 (1984)

    Google Scholar 

  65. Latham, G.A.: Existence of EMS solutions and a priori estimates. SIAM J. Matrix Anal. Appl. 16, 943–953 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  66. Levitan, E., Chan, M., Herman, G.T.: Image-modeling Gibbs priors. Graph. Models Image Process. 57, 117–130 (1995)

    Article  Google Scholar 

  67. Lewitt, R.M., Muehllehner, G.: Accelerated iterative reconstruction in PET and TOFPET. IEEE Trans. Med. Imaging 5, 16–22 (1986)

    Article  Google Scholar 

  68. Liu, C., Rubin, H.: The ECME algorithm: a simple extension of EM and ECM with faster monotone convergence. Biometrika 81, 633–648 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  69. Llacer, J., Veklerov, E.: Feasible images and practical stopping rules for iterative algorithms in emission tomography. IEEE Trans. Med. Imaging 8, 186–193 (1989)

    Article  Google Scholar 

  70. Lucy, L.B.: An iterative technique for the rectification of observed distributions. Astron. J. 79, 745–754 (1974)

    Article  Google Scholar 

  71. McLachlan, G.J., Krishnan, T.: The EM Algorithm and Its Extensions. Wiley, Hoboken (2008)

    Book  Google Scholar 

  72. Meidunas, E.: Re-scaled block iterative expectation maximization maximum likelihood (RBI-EMML) abundance estimation and sub-pixel material identification in hyperspectral imagery. MS thesis, Department of Electrical Engineering, University of Massachusetts Lowell (2001)

    Google Scholar 

  73. Miller, M.I., Roysam, B.: Bayesian image reconstruction for emission tomography incorporating Good’s roughness prior on massively parallel processors. Proc. Natl. Acad. Sci. U.S.A. 88, 3223–3227 (1991)

    Article  Google Scholar 

  74. Mülthei, H.N., Schorr, B.: On an iterative method for a class of integral equations of the first kind. Math. Methods Appl. Sci. 9, 137–168 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  75. Mülthei, H.N., Schorr, B.: On properties of the iterative maximum likelihood reconstruction method. Math. Methods Appl. Sci. 11, 331–342 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  76. Nielsen, S.F.: The stochastic EM algorithm: estimation and asymptotic results. Bernoulli 6, 457–489 (2006)

    Article  Google Scholar 

  77. Parra, L., Barrett, H.: List-mode likelihood: EM algorithm and image quality estimation demonstrated on 2-D PET. IEEE Trans. Med. Imaging 17, 228–235 (1998)

    Article  Google Scholar 

  78. Penczek, P., Zhu, J., Schroeder, R., Frank, J.: Three-dimensional reconstruction with contrast transfer function compensation. Scanning Microsc. 11, 147–154 (1997)

    Google Scholar 

  79. Redner, R.A., Walker, H.F.: Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 26, 195–239 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  80. Resmerita, E., Engl, H.W., Iusem, A.N.: The expectation-maximization algorithm for ill-posed integral equations: a convergence analysis. Inverse Probl. 23, 2575–2588 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  81. Richardson, W.H.: Bayesian based iterative method of image restoration. J. Opt. Soc. Am. 62, 55–59 (1972)

    Article  Google Scholar 

  82. Rockmore, A., Macovski, A.: A maximum likelihood approach to emission image reconstruction from projections. IEEE Trans. Nucl. Sci. 23, 1428–1432 (1976)

    Article  Google Scholar 

  83. Scheres, S.H.W., Gao, H.X., Valle, M., Herman, G.T., Eggermont, P.P.B., Frank, J., Carazo, J.-M.: Disentangling conformational states of macromolecules in 3D-EM through likelihood optimization. Nat. Methods 4, 27–29 (2007)

    Article  Google Scholar 

  84. Scheres, S.H.W., Núñez-Ramírez, R., Gómez-Llorente, Y., San Martín, C., Eggermont, P.P.B., Carazo, J.-M.: Modeling experimental image formation for likelihood-based classification of electron microscopy. Structure 15, 1167–1177 (2007)

    Article  Google Scholar 

  85. Scheres, S.H.W., Valle, M., Núñez, R., Sorzano, C.O.S., Marabini, R., Herman, G.T., Carazo, J.-M.: Maximum-likelihood multi-reference refinement for electron microscopy images. J. Mol. Biol. 348, 139–149 (2005)

    Article  Google Scholar 

  86. Schmidlin, P.: Iterative separation of tomographic scintigrams. Nuklearmedizin 11, 1–16 (1972)

    Google Scholar 

  87. Setzer, S., Steidl, G., Teuber, T.: Deblurring Poissonian images by split Bregman techniques. J. Vis. Commun. Image Represent. 21, 193–199 (2010)

    Article  Google Scholar 

  88. Shepp, L.A., Vardi, Y.: Maximum likelihood reconstruction in emission tomography. IEEE Trans. Med. Imaging 1, 113–122 (1982)

    Article  Google Scholar 

  89. Sigworth, F.J.: A maximum-likelihood approach to single-particle image refinement. J. Struct. Biol. 122, 328–339 (1998)

    Article  Google Scholar 

  90. Silverman, B.W., Jones, M.C., Wilson, J.D., Nychka, D.W.: A smoothed EM algorithm approach to indirect estimation problems, with particular reference to stereology and emission tomography (with discussion). J. R. Stat. Soc. B 52, 271–324 (1990)

    MATH  MathSciNet  Google Scholar 

  91. Sun, Y., Walker, J.G.: Maximum likelihood data inversion for photon correlation spectroscopy. Meas. Sci. Technol. 19(115302), 8 (2008)

    Google Scholar 

  92. Tanaka, E., Kudo, H.: Optimal relaxation parameters of DRAMA (dynamic RAMLA) aiming at one-pass image reconstruction for 3D-PET. Phys. Med. Biol. 55, 2917–2939 (2010)

    Article  Google Scholar 

  93. Tarasko, M.Z.: On a method for solution of the linear system with stochastic matrices (in Russian), Report Physics and Energetics Institute, Obninsk PEI-156 (1969)

    Google Scholar 

  94. Trummer, M.R.: A note on the ART of relaxation. Computing 33, 349–352 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  95. van der Sluis, A., van der Vorst, H.A.: SIRT- and CG-type methods for the iterative solution of sparse linear least-squares problems. Linear algebra in image reconstruction from projections. Linear Algebra Appl. 130, 257–303 (1990)

    MATH  Google Scholar 

  96. Vardi, Y., Shepp, L.A., Kaufman, L.: A statistical model for positron emission tomography (with discussion). J. Am. Stat. Assoc. 80, 8–38 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  97. Wernick, M., Aarsvold, J.: Emission Tomography: The Fundamentals of PET and SPECT. Elsevier Academic, San Diego (2004)

    Google Scholar 

  98. Wu, C.F.J.: On the convergence properties of the EM algorithm. Ann. Stat. 11, 95–103 (1983)

    Article  MATH  Google Scholar 

  99. Yu, S., Latham, G.A., Anderssen, R.S.: Stabilizing properties of maximum penalized likelihood estimation for additive Poisson regression. Inverse Probl. 10, 1199–1209 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  100. Yuan, J., Yu, J.: Median-prior tomography reconstruction combined with nonlinear anisotropic diffusion filtering. J. Opt. Soc. Am. A 24, 1026–1033 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, University of Massachusetts Lowell, Lowell, USA

    Charles Byrne

  2. Food and Resource Economics, University of Delaware, Newark, DE, USA

    Paul P. B. Eggermont

Authors
  1. Charles Byrne
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul P. B. Eggermont
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles Byrne .

Editor information

Editors and Affiliations

  1. Computational Science Center, University of Vienna, Vienna, Austria

    Otmar Scherzer

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media New York

About this entry

Cite this entry

Byrne, C., Eggermont, P.P.B. (2015). EM Algorithms. In: Scherzer, O. (eds) Handbook of Mathematical Methods in Imaging. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-0790-8_8

Download citation

Publish with us

Policies and ethics

Search

Navigation